New coworker trying to catch me out

I started a new job last week and when chocolate was being offered around the office it was the time for me to say "no thanks, I'm actually vegan."

A coworker immediately came over and looked at my feet saying "just checking that you're not wearing leather shoes or anything", which I wasn't.

Might sound simple but it really frustrates me when as soon as someone learns you're vegan that they try and catch you out as if to make themselves feel better. It's the same with being given hypothetical scenarios where the person is trying to design them to make you admit you'd break your veganism. Anyone else?

Edit: Loving the classic vegan debate happening below but feel like we're missing the point. I was focusing on how often other vegans experience people trying to catch them out. It's not about whether offices should provide vegan treats or whether I should or should not "announce" I'm vegan.