There is a MAJOR difference between being arrogant, and being insane, genocidal, maniacs. EVERY race is arrogant to some extent, have you played Morrowind? The dunmer in that game were INCREDIBLY arrogant beyond belief, but very few of them were evil in any way.
And as everyoen here is pointing out, not all Altmer are Thalmor. That is like saing all Germans were Nazi's, or that all Nords are recist, bigoted, stormcloaks.