When you backup for the first time, it will need longer than afterwards, because it's backing up everything. Those made afterwards need less time, because iCloud just saves what changed since the last backup.
You can deselect apps that don't need to be backed up. Go to Settings - General - Storage & iCloud Usage - iCloud - Manage Storage - select your device (if there are more than one Apple devices), then disable all the apps that shouldn't be backed up.
The backup will need less iCloud storage afterwards, and the backup will need less time.