Honestly doesn't bother me, they hate, they hate who cares. I'm not hurting anyone, not committing crimes... Am I? No. I'm not so be the mature one and brush it off and give haters the bird and be yourself. They can't change who we are. That's how I think of it at least.
And yes there is 'that side' to the fandom where there is mature things which most know as Yiff which are usually pointed out first and used against us but honestly they can think whatever they want, at least you know what your doing, right?
It's just simple society... Others like to stay in the crowd to fit in as some like to be thereselves and there's no problem with that.