The iPhone 4s is available quite cheaply because it was first introduced back in 2011. It will almost certainly not receive the iOS 9 update this fall. After that point, you'll find that a growing number of apps will no longer be available to devices on iOS 8. The iPhone 4s was discontinued 2 years ago and could be as much as 4 years old. If it still has its original battery, it will likely not be able to hold a full charge any longer. I wouldn't recommend the iPhone 4s as a first iPhone at this time.
The iPhone 5s is still available new with up to 32 gb of storage and as it is a newer design by 2 years it will be supported for longer than the older iPhone 4s.
Additionally, the faster LTE data service is not available for the iPhone 4s.
If you have a limited budget, There are numerous sellers, large and small, who sell used iPhones. Make sure to do your research before buying a used iPhone from a stranger.
Before buying a used iPhone please read this thread, which will help you avoid most of the risk involved in buying a used iPhone.
Before buying a new second hand device online Apple iPhone Forum