http://www.imperial-library.info/content/battlespire-spirit-daedra says "Dremora have long served Dagon... But not always so."
So, what if this doesn't necessarily mean that the Dremora had a different master (or no master at all), but rather that they've always served Dagon, but they also happened to serve him before he became Dagon?
It's also possible that they have a similar function as the Jills of Akatosh (preservers) and Alduin's dragons (destroyers), in that, when their master switches personalities, so do they.
So, am I onto something, or am I just talking crazy talk?