Never say never. Apple invented, designed, coded the iPhone. Only they know for certain what can or can't be done with any aspect of the iPhone. We don't have the sandbox to rip apart the iOS firmware to verify anything.
Many years ago in the xbox360 community, all the expert coders said it was impossible for Microsoft to access, let alone load new firmware to the 360's DVD drive. I challenged these so called experts stating that Microsoft built the thing and knows much more than anyone else, so writing new firmware to the DVD drive could be done because they have the unlock key for the drive. I got banned from a few forums for stating this and challenging the coders. About a year later, Microsoft introduced new dashboard which wrote new firmware to the DVD drive and wiped out everyone "hacked" DVD drive with "factory unhacked" firmware. Microsoft did what even the expert coders had been saying for years was impossible.
Now I'm not saying it's possible or impossible for the fingerprint data stored in the phone to be accessed from an outside source like Apple or some government agency. The fact is we simply don't know exactly how everything in the iPhone works and interacts with other entities or processes, whether internal or external to the iPhone.
Apple has all the crypto keys for our phones internal components. They have to have these crypto keys incase something needs updating to fix a bug. How strong is the crypto? 40 bytes? 256 bytes? If they are using a 40 byte key, then even an older computer could break the key in hours to weeks - assuming you have apples firmware sandbox and know about crypto, prime numbers, etc.
If your concerned about others accessing your fingerprint, don't use that feature or use another part of your hand to unlock the Touch ID.