Jagar Tharn?

Whoever the emperor was at the beginning of Oblivion I haven't played in a while

The emperor at the beginning of Oblivion was the same emperor as the first three games: Uriel Septim VII.

Although (fun fact): the Arena intro mistakenly refers to him as Uriel Septim IV.

I would say the decline began with the simalcrum but that the oblivion crisis did far more damage than any of the others did. The Empire has traditionally been driven by man for better or worse. If mankind wants to avoid being Elven slaves once again then they need to unite. this seems unlikely as the Redguards have been abandoned by the Empire and the Nords are currently tearing themselves apart.

To reverse this I think the new emperor (provided Mede II is canonically dead) will have to reconnect some ties with Hammerfell, not by trying to make them a province again but by hopefully securing a defensive alliance with them so the Dominon will at least have to fight them both at once. The same could be tried with Argonia but I bet they would be less willing to cooperate. I would also offer provincial status to the orcs (if they do not already have it) as this would give them more of an incentive to help and support the Empire. Honestly if Skyrim becomes independent than I think the Empire will be completely doomed as Cyrodiil and High Rock do not boarder one another and thus will be isolated. This will only encourage a traditionally divided High Rock to break away.

I don't expect the Empire to recover, at least not in TES VI. I think VI will either take place during the Second Great War with the Empire outmatched by the AD or the second Interregnum after it has already collapsed. Perhaps another Empire will emerge in a later game but the trend of the games has been its weakening so I suspect that it will collapse.

Personally, after five games I'm okay with the Empire collapsing and the Dominion becoming the defacto rule of law in the world. I'm really curious how the Dominion (and the Thalmor) are seen from within.

Life in the Dominion (maybe) could be compared to life in Stalin's Russia / Nazi Germany / Mao's China?

I think that's how Bethesda is trying to set it up with the Elven racial superiority and the "purges" of political dissenters that occur in Valenwood. It seems very totalitarian and the Thalmor seem to be like the Nazi party. I have no doubt that they will turn men into second class citizens and there is even a chance that they may try to enslave some of them like the Ayleids did. That would be a very interesting and different climate for TES VI.

I, as an Orc (possibly one other race that they wish to purge) would help free thoses, even on fear of death

Yeah, I can definitely agree that the Thalmor have a prominent nazi vibe about them.

I think that before the Simulacrum, the cracks began to form, but I don't think it was until the Oblivion Crisis that the cracks became big enough to where the empire began to explicitly weaken. By the time of the Nerevarine, the cracks were probably big enough to where only the astute or highly intuitive could notice them. Oblivion was definitely the point of no return, however.

I expect that the likes of Baladas had been saying the same thing about outliving the Empire for centuries. They don't care enough to follow internal Imperial politics.