I'm That Guy

When I'm being sarcastic to people I say something like oh sweetie or hun or whatever. Guys are always like are you flirting with me? Then I have to explain. Such a awkward conversation.
Am I the only one this happens to?

Oh please, he's from the South, everyone is Sweetie or honey or darlin'.

sweety is for everyone