Because he betrayed the Queen of Hearts and married the Rottweiler. That's a big enough reason for quite a few people out there.
I agree Lieblich. If he can lie to his ex wife, sneak around and cheat on her then what will he do too his country. Some people can not forget what he has done in the past and turn the other cheek and look away. He's doing what is asked of him no more no less, he's a prince doing his job just like every other royal out their IMO.