Is it selfish to avoid my boyfriends friends?

The past 2 years that I have been dating my boyfriend I have always felt a little left out by his group of friends. They have never been openly mean to me but by their silence (especially his best guy friend, Joe) leads me to believe that they wish I were someone else. I am three years younger than all of them and I'm guessing that is the original source of dislike from them, but my bf insists that no one has a problem with me, even though their opinion of me hurt me a lot. He has a female best friend who has a bf that flirts with all of them innocently and she calls my bf pet names. It bothered me so much that I refused to be around any of them for almost a year but this is not the real problem. Is it selfish to avoid my boyfriends friends? All they do is drink. Two of them are legitimate alcoholics (one of them blew 700 dollars on beer in 5 days) It makes me want to cry every time I watch my bf's female friend hug him and call him lovey (she texts him I love you several times a week) and take pictures of herself in his clothing. ( I blocked her on facebook) A part of me feels bad bc the girl is a nice girl she just needs to learn her limits and when I confronted her about it she was reasonable but would not change her ways. It is akward when I see his friends now because I know that they don't really want me there and my bf doesnt see that because when "I asked them (his friends) they all said they didnt have a problem with you (me)" Obviously! Why would they tell him they dont like his gf? I'm not asking him to not spend time with them. I just dont want to listen to them talk about fishing and the firehouse and drink as his female friend flirts w him in front of me! So I avoid them. They never liked me yet my bf insists that I'm being selfish by not wanting to be a part of his social life. Last I checked being a little socially akward is not a reason to dislike someone. My bf has encouraged me to speak more when I'm with his friends even tho I've told him I really really don't want to be around them and now I just avoid them and brush off his invitations to go out. Joe's gf was accepted into the group w/o a problem and we were friends until she stopped returning my calls out of no where and that is another reason why I don't want to be around his friends. There are maybe 6 or 7 of them and all of them are girls except for Joes gf and my bf's female friend, so its very uncomfortable to go out with them. Quite frankly I'm disappointed that I am not friends with either of them, but they made up their minds about me before I had a chance. My bf and I cannot reach a compromise and it could easily cause a brake up. Who is being unreasonable and what should be done?