iPhone 4s

The iPhone 4s is available quite cheaply because it was first introduced back in 2011. It will almost certainly not receive the iOS 9 update this fall. After that point, you'll find that a growing number of apps will no longer be available to devices on iOS 8. The iPhone 4s was discontinued 2 years ago and could be as much as 4 years old. If it still has its original battery, it will likely not be able to hold a full charge any longer. I wouldn't recommend the iPhone 4s as a first iPhone at this time.

The iPhone 5s is still available new with up to 32 gb of storage and as it is a newer design by 2 years it will be supported for longer than the older iPhone 4s.

Additionally, the faster LTE data service is not available for the iPhone 4s.

If you have a limited budget, There are numerous sellers, large and small, who sell used iPhones. Make sure to do your research before buying a used iPhone from a stranger.

Before buying a used iPhone please read this thread, which will help you avoid most of the risk involved in buying a used iPhone. Before buying a new second hand device online Apple iPhone Forum

 

The best place to go, although not the cheapest, is an Apple Store or an authorized repair centre. Independent repair shops are cheaper but you should make sure that they use genuine Apple replacement parts.
The price that Apple charges to replace an iPhone battery, if nothing else is wrong with the iPhone, is $79.00. iPhone Battery and Power - Apple Support

 

Which apps are you trying to remove? The stock apps that come with the iPhone cannot be removed and will not show the X when you push and hold the icons.

 

Have you checked in Settings>General>Restrictions>Deleting Apps and made sure that the switch is turned on(showing green)? If the switch is turned off, apps cannot be deleted from the iPhone.