I take the view of the social contract: if they won't respect the rules and are impolite to you, they forfeit their right to politeness back. You're perfectly justified in hanging up on someone if they're so rude as that.
It's also true they're just doing their job, but that just makes it more okay to hang up on them. If anything, you're helping them hit quota by letting them move on quickly. It's a service!