Lesser daedra are the "chosen form" of daedra who don't have realms. To clarify, all golden saints are part of one being, which i'll call "GiantGoldenSaint" for the example's purpose. When you banish one of the forms of the daedra, that piece of it goes back to oblivion. However, in order to exist in various forms, GiantGoldenSaint doen't have a realm. This is also why they go through the dreamsleeve regularly, they have no home to protect them from it. This would also explain the "clan" aspect of Daedric life: The clans are all part of one whole, and therefore are bound to each other. This is why GiantGoldenSaint has many forms who have changed who they are in service too, and yet no Golden saints fight. Why would one of your arms fight another? However, GiantGoldenSaint has some beef with GiantDarkSeducer. They can swich who they serve because they aren't bound to there masters.
Some lesser Daedra chose their forms with specific masters in mind, for example GiantNocturnal chose his form to service the Daedra Prince Nocturnal. Nocturnal's realm is lightless, and GiantNocturnal's form can see in the dark.
This theory boils down to that Daedra have two choices: A.) stay together and become one really [censored] super-dude or B.) Split myself into little pieces so that I can have massive numbers of weaker selves. The larger the number of pieces GiantGoldenSaint splits him/herself into, the weaker the individual will be. This is why there are fewer of the stronger Daedra.
I am wide open to criticism on this theory, and I can myself find some minor flaws.
Edit: Clarification and grammar