By the time of Infernal City, Argonia is under the control of the An-Xileel faction, which is said during the Oblivion Crisis to have flooded into Oblivion with such great force and strength that Dagon's military leaders closed the gates, this may be propaganda though.
Actually, someone brought up how the other provinces fared during the Oblivion Crisis. I gave a silly scenario of what happened in BM (mainly disease, and bugs often killing cultists, and guerrilla warfare), and MK alluded that I was not that far off.
Also, lets look at the misfortune the dunmer faced during the end years of the 3rd era.
1) The Tribunal was cast down, removing the god shield they had
2) Nords and orcs raided Redoran territories bordering Skyrim before the Crisis
3) House Indoril being picked apart by the other houses and Helseth
4) The military arm of the dunmer is in decline (house Redoran) and faced with bad corruption
5) The Oblivion Crisis ravages Morrowind a new one. Redoran lost badly on the new capital, Ald'rhun, even with the aid of the giant undead emperor crab. Telvanni was really the only house putting up a fight, but even then they weren't really gaining any ground
6) The Ministry of Truth falls, Red Mountain blows its top, annihilating Vvardenfell and covering it with a foot or 2 of ash and causing the seas to boil, even 40ish years after Red Mountain blew its top.
Really, this was the perfect opportunity for the Hist and argonians to strike back.