It always sucks having to choose between friends and something you like. But, the only way to be happy is to do what you like, and make new friends that share your interests. That's how I've always seen it.
In high school I developed sort of a defense. I realized that anyone, and everyone, else's opinion only matters if I care what they think. There is no reason to care at all about what someone thinks, unless you choose to. If someone dislikes something that you like to do, let them know that you aren't going to stop doing something that you enjoy because of them. But, you won't go an about it and rub it in their face. If they don't like you for that, then it's time to go in search of new friends. Find people who, if they don't accept everything about you, don't attempt to push you our of something that you like to do because they don't like it.
I learned very early to be no one other than ME, and to do what I enjoy. Everyone else's thoughts on this don't matter. Eventually, I found friends who accepted everything about me. As quirky and wierd as some of it may be to them, they still care about me.
In essence, bear through the haters, and try not to let what they think get to you. After all, they are one of more than 7 billion people. How much importance is one person in your life if they don't like what you do? You will, in all probability, see them one day, and then never see them again for the rest of your life. It happens a lot.