Quantity versus quality. Dumner make glass armour and live in giant insects; they also make netch leather armour and live in shacks. Most Orsimer in Skyrim don't wear Orcish armour. We've never been to Alinor, so we really don't know what common Altmer material culture is like.
Also, good luck hiring a Telvanni to grow you a tower, or goading a gaint crab to die just where you need it. Responsible Architecture doesn't seem like something that could be easily transplanted.
Bolded the important bit. When I explain how Romans beat the barbarians to people despite the barbarians even today having a reputation as fearsome warriors, it's because they were fearsome individually, but that wasn't enough against the mass of cold, hard Roman discipline.
Move that to tamriel. Imperials have a reputation as great persuaders and unifiers. In Morrowind, other soldiers won't even speak to an out-of-uniform legionnaire. Cold, hard discipline trumped the superior equipment of the various individuals they fought. Where it didn't, they hit you with a robot.