Powerful magic. We are not talking mortal magic here. We are talking about powerful Daedric magic.
I'd like to emphasize that this is not true at all.
The quote from Darkest Darkness explains it already - summons stay here indefinitely, unless they are
told to go back by the summoner. The duration of summon spells in all the games is merely a gameplay thing. Remember the sorcerer apprentice in... Ald'ruhn, I believe? He summoned a scamp, but wasn't able to control it, which caused the scamp to destory his home and stay there until the player eventually got rid of it.
It must be due to them just not being known about in Cyrodiil. The SI seem to be more like Morrowind in some ways.
And sorry, this is no personal grudge against you or anything, but that is not true either. The true reason is simply that daedra worship has a history with the Dunmer, and the daedra we find in the game are especially those of the four daedra who are considered to be evil by the Dunmer (Molag Bal, Sheogorath, Mehrunes Dagon, Malacath).