If you talk to an Altmer that lives in the city, she states that if the Dunmer try and be hardworking like everyone else, they would be accepted. That's coming from an ALTMER. Also, the only racist ones in the city are the two drunks.
Conversely, there are a bunch of hard-working Argonians at the docks who are treated as total scum and get paid far less than Nord workers. Brunwulf Free-Winter also speaks how Ulfric doesn't give a [censored] about non-Nords being harassed by bandits on the road.
Ulfric is either a Nord supremacist bigot or he is a really, really crappy Jarl that doesn't know what's going on in his city.