I have this problem too.
Here's a workaround: dial your phone and LEAVE a voicemail message (yes leave one!) Then dial back and delete the voicemail you just left. That clears the red dot.
Two other observations when my voicemail is empty:
1) When I reboot the phone, the red dot...