a chunk of the lizard population migrates north, every other race fills the void because its the only place the thalmor cant touch.
This could really be a great setting. One hundred years after skyrim, the thalmor are expanding. Elswyr is conquered, as well as parts of cyrodil and hammerfell. High Rock, Hammerfell, Cyrodil, and Skyrim have devolved into nearly powerless bickering states.
Black Marsh has swelled with a population of refugees and rebels, people from all countries, who think their leaders are useless, and who hope to make a united front against the Thalmor. Aldmer, unfortunately, now get the treatment and hatred that used to be given to Orcs.
Black Marsh would still have a majority of Argonians, but there would be plenty of excuse to focus on human-types and their settlements. To reassure the suits at Zenimax.