A lot of people don't believe what Camoran claims. It is highly debatable, but that isn't the topic of the discussion.
Maybe so, but sometimes to get at what's going on with ES it is useful to talk around various topics
Manky was a pawn of Dagon - read The Refugees in TIL - and he was not only not Ayelied - his mother was a Bosmer and considered him to be a monster and she saw his father Haymon Camoran (who by repute was part Breton) likewise ... as did the rest of Tamriel.
Both Father and Son tried to conquer areas Tamriel using Daedra - the father raised a gigantic undead army too. both committed atrocities at Kvatch. I hope you will forgive me for seeing their allegience as a sign of twisted mentalities - given that Dagon and his servants look on all mortals as prey to be hunted for their personal amusemant.
I must admit that I do not remember hearing of any towers in other parts of Nirn - but there is relavent Lore in the Imperial Library about the Leaping Demon King who can leap no more. Apparently he and 'The Greedy Man' teamed up to add to Nirn so that Alduin the World-eater would burst from overeating and thus Nirn would be freed from the repetative cycles that seem to be its lot.
The story goes that Alduin discovered this and cursed The Leaping Demon King, who thenceforth became known as Dagon the Deadric Prince of Destruction, to leap no more until he had returned all the bits that he had added to wherever they came from. So if this is both Lore and fact then it might be that Tamriel (or part of it) is one of those added bits that keep Dagon imprisoned ... and dagon is undoing it one bit at a time