The fact that Titus Mede got things together (mostly) 150 years before Skyrim and that their are STILL Imperials to the extent that Skyrim is about to break into a civil war extends two things.

1. The Imperial presense in Skyrim is low, the Empire is in trouble/flourishing but are hated, but part of Skyrim wants to stick with it.

2. The Empire is in fact very strong and is influencing Skyrim to the point that there is a civil war whether or not to secede