The "Photo Library" is simply pointers to all photos on the phone. Hence the "Photo Library" doesn't take up any space on the phone and you can not directly delete from it.
There are two types of "Folders" (which are called albums). The ones you create from Photo's already on the phone and the ones you created from iTunes.
Any album you create in iTunes and transfer to the phone must be deleted in iTunes and re-synced.
Any album you create on the phone from existing pictures can be deleted the same way as you did for the Photo Stream, except you open the Album you want then proceed with the last three steps.