Well it's no sillier than Aedra worship. Sure the nine have alters to cure disease, but in actual practice the daedra give their worshipers a lot more personal attention.
That's the thing. Aedra worship is generally associated with good, and is the mainstream religion. People follow it because their parents did it, it's tradition, and that's how things are done. They might not think that much about what they are doing, especially not in a "medieval" world, where the gods actually exist.
And what is that about "personal attention"? Do they have deep conversations? Eat dinner together? No. I doubt the Daedra would care much about the people who hang around their statues much - not when there's people like the Nerevarine, the Champion or Cyrus around. Azura didn't even care enough about her followers to save them from their vampirism. I mean, could you really imagine someone praying to Malacath or Boethiah for a good harvest? Healthy children? You go to the Daedra Princes if you want a large favour, and they usually charge you a high price for it. It's not a beneficial relationship for the "believer". It's not something to build upon.
Sure, there certainly must be cults and such who has the "follow me and reach paradise-agenda", but those are sects, not societies.
Or. I've seen to be foiled by my own argument about Aedra. If a society were built around Daedra instead of Aedra, it would still be the same thing - it wouldn't matter to most people. You still need to live, and people would do what they need to do because that is how things are done. They wouldn't get much more out if it, though, unless they worshipped one of the (seemingly) more benevolent Daedra.
You can go ahead and disregard most of my post now.
Which makes me wonder at times why everyone kept remarking "How could the Nine let this happen?" regarding Kvatch, when there's not much indication that the Nine Divines actually do anything beyond shrine blessings. Are the people living in the heart of the Empire really that dense?
Did you miss the whole Oblivion ending?
Granted, Kvatch happened before that.