Why are some straight people so VERY scared of people “looking gay”?

There’s two people I know whose fear have started to sound irrational to me. Been out of the closet for 10 years, I look like a butch lesbian and am usually immediately introduced to the queer community the moment I enter a new area because of it. These two people are so against the idea of even assuming anyone’s queer that they’ve started saying to my face they even think I’m straight? Like….I genuinely don’t know what wavelength they are on by now. But I’ve seen a similar fear among a handful of other strictly straight people, even if they call themselves allies and are for gay rights. But it just comes off as iffy to me. They are aware you are allowed to “look gay” and be straight? It’s like asking if someone who looks like a metalhead likes metal music. But apparently this comparison har just made them double down even more. I dunno what to do or whether to be concerned.