For myself, I think I could say a lot about this, as in our RPGing group, we have sort of "private jokes" about that we are considered weird by others. While in real, we are not. Although it's not only about ourselves, we also say this about the people who go on all sorts of fantasy actions, and the most about people who dress up in fantasy costumes and such, and in large sense about the people who go on these "live battles" (I would like to know, is there a special word for it in English?), but again, that's because most of us count ourselves among these groups. I am not sure how well can I express it, but for example: recently we were returning from our summer cottage where we gathered for playing Dungeons&Dragons, and so we went back, sat in the bus, and one of my friends took out the rulebook and started to study some spells. The rest of us looked at each other, saying: "That's horrible!" While of course we didn't mean it, but it's simply became a "private joke" of our company. Similar things happen when we see people dressed up in fantasy costumes in the metro or such things. We usually start to talk between ourselves, as if we were speaking to the others: "Hey, you are going to the Battle of Five Armies? Are you? Yes, of course you do! You must know good old [insert a fellow geek's name]! Of course!" But most typical, when we are for example arranging an RPG meeting, we joke like "Yes, let's go and play. I've been missing it! I just got through the exams for this semester and now I finally have time for playing! Of course! Yes, of course it's horrible! Of course I've not been studying at all, but I've been planning my character! When are we going to play? Next weekend? I'm going to be on my friend's cottage just before that. Of course I will take the handbook with me! Of course! I know I'm an addict! Yes, and I am proud of it! Which is horrible!" And all of this with the imaginary "
".
So, in my case, I don't see being seen as weird as a problem coming from the "outside" (and I don't think I ever encountered anything more than people not understanding what the things we do are), simply because my friends - for some reason, and it reaches into times long ago - made it a joke from the "inside".