My favourite races are Dunmer, then Altmer, and then Nords. In all three cases, I wouldn't join the Empire. At least, not the Mede dynasty's Empire.
If the Empire was still ruled by the Septim dynasty, I would support it. But an institution without Divine authority is worthless - just a profane political hegemony with no special meaning. The only legitimacy the Empire ever possessed was thanks to Akatosh and Talos. The Dragonborn Emperors are a thing of the past, so Akatosh no longer has any ties to the Empire. On top of that, the Empire has forsaken Talos, so what remains to legitimise the Empire? Nothing but worldly power, which can be broken. The Aldmeri Dominion is ascending, and the Fourth Age will be an Anuic one, with the elves returning to a position of strength.
Without Divine patronage, the Empire is worthless. The Stormcloaks understand this, which is why Skyrim will be stronger under Ulfric in the long run. I suspect an apocalyptic war between Men and Elves will come, and if the Stormcloaks lose the civil war, Talos worship will eventually be stamped out, which will result in Talos losing his divinity, which ultimately means that Men will be very weak in the final struggle.