This is an old post to which i'm replying, and i'm not sure if someone already shared the answer but i discovered it only today. Its annoying to see the voicemail notification which will never go even with a restart. I thought it was a virus as my provider does not offer voicemail services nor have i configured any number to connect to a voicemail server. It started showing up when I left my mobile internet on over night.
-- go to app > phone > and click 'force stop'. you can launch the app after that but it should clear the voicemail notification icon
