Most of the apps ask for access in order that they can work properly. Instagram or Flickr e.g. need permission to access your Photos library, when you want to add a picture stored in the Photos app for the first time. Others need to access Photos so that you can save pictures to the Photos app.
These apps won't act without you knowing it.
If you think that an app shouldn't be able to access your images, don't give access, or disable the app in Settings - Privacy - Photos.