None of this would interest the daedra in any significant way except for the fact that they want all people and their fortifications gone.
Why? Daedra don't come in to conquer and settle the land. Frankly, they really can't come into the Mundus unless called or through special means (Sotha Sil's arrangement with the Daedric Princes, the Dragonfires). The Daedra don't want to destroy everything for the sake of it. Sure, Dagon likes stomping on people, but I don't picture him as the type to hunt down the last few survivors - he mostly appears, destroys stuff, and then leaves. That's what he does; his sphere is destruction. I don't really understand why the lesser daedra would give a flying poo about what happens on Nirn, other than scratching their heads at the whole 'mortality' and 'death' thing.
Ghostgate, is a GATE. It isn't some huge fortress. And I doubt the Bouyant Armiger's force is comparable to the entire imperial legion - really, they're just an elite group of fighters, not a standing army.
I guess I find it confusing why TES keeps using Dagon as some big bad boogey man that wants to trash Nirn - as if he's got some personal grudge or something against it. I mean, he might have been behind Tharn's actions in Arena, and he was definitely behind the whole Battlespire thing - is he just trying to destroy the Towers that prevent the creatia in Nirn from dribbling into Oblivion? Is he after the Towers?