I think it depends the higher ranking bosmer likely are allowed simply to maintain control the others are just cannon fodder and not "True Elves". I think it's partly because Bosmer have interbreed with men quite a lot over the years and the Thalmor don't like that idea at all. Which if some of the theory's on the origin of goblins are correct is completely understandable and makes it easy to see why they wouldn't want that idea to spread to the Altmer.

Altmer hate all the other mer, except for Bosmer. If the Falmer were still pure, they'd be best of kin.

Bosmer love everyone.

Dunmer hate everyone.

Orcs don't have time to hate or love. They are too busy minding their own business.

At least until Daggerfal, wayrest and sentinel come along and sack it every couple of centuries.

The Dunmer and Altmer hate each other. The heretic's worse than the infidel, and all that.

Daggerfall, Wayrest and Sentinel are only three cities with their own agenda's though.

From what I remembered of the dialogue with Orc npcs in Morrowind, it doesn't seem like they hate the Redguards or Bretons, though they do seem to think Breton's are cowardly.

ESO backs that up as well, at the time of the game even one of the more hostile clans seems to allow the odd outsider inside the stronghold as long as they earn their keep.

I read this thread title and immediately thought of my british friend who once asked if the Chinese and Japanese get along, got the answer "hell no are you kidding me?" and then proceeded to rebuttal "well I thought they would all get along because they're all asian."