I love the Empire. In my opinion, they should rebuild and reconquer everyone. Destroying all pathetic hopes for freedom and liberty and snatching victory tfrom the fetid jaws of the "provinces". For the Empire!
Wasn't the old Empire more about keeping order than conquering?
I'm very interested to see how the Empire looks as of Skyrim. We might have a better idea once the next novel is out. If it's born from a legacy from the protagonists of that book it's probably a more goodly Empire. If not it's a big question mark.
I like that the Empire outlawed slavery. What's interesting is whether or not I support the empire is basically the polar opposite of your reason, though parallel in some ways. I want the Empire in control to keep order and enforce freedoms like the ban on slavery.
I very much think the Aldmeri Dominion will be a grave threat down the road too, for the simple reason that its goal is not the good of all races, not even the good of all mer races, but instead its goal is the dominion of the Altmer over all of Tamriel. It's quite a villainous situation in my opinion, and a revitalized Empire would provide a counter to its power.
Black Marsh is a big question mark, and I suspect the second book will reveal more.
Spoiler But if I remember the first book right it seems like some not-so-nice rogue Hist may have taken control of the Argonian society. I don't remember though.
I think Elswyer would be more worried about its internal issues but lots of time passed since The Infernal City.