Because elitists are skilled (or sometimes not) players who are jerks to anyone that isn't as good as them. That is what elitist means. They look down on people who aren't like them.
Now if you are talking about hardcoe players and casuals, that is a better question. Unfortunately most hardcoe players that stand out are the elitists, and so they give them all a bad name among casuals. And the most noticeable casuals are the ones who really svck, giving them a bad rep among hardcoes.
Kind of like how everyone complaining about a game gets heard, and gives the game a bad rep, while the much higher number of people who like it don't get heard so much. Probably because they are playing it instead of complaining.