The Soul Cairn as seen in Dawnguard has a bit of a juxtaposition... On one hand, we have really dark gothic architecture. On the other, we have the weird otherworldy growths (like the glowing craters, or the strange rock outcroppings). And then there's the blackened plants everywhere.

I think the Soul Cairn is a fragment of a previous Kalpa stashed away by the Leaper Demon King that became Mehrunes Dagon once Alduin caught him in the act. The Ideal Masters later came across it, seemingly abandoned by Dagon, and made it their home, doing a bit of redecorating in the process.

And then the Battlespire invasion occurs. Dagon needs a staging area, remembers that little bit he left behind before Alduin had his way with him, comes in, and sees these abstract entities styling themselves "Ideal Masters". Feh. Frickin' squatters.

But then, who are the Ideal Masters? Previous Kalpic beings who discovered soul collecting as a means of kalpic sustenance or something?

I assumed the 'Ideal Masters' were powerful mages who created soul gems. Also, how feasible would it be to create a pocket realm of your own in oblivion? Perhaps these ideal masters made a deal with a Prince. One prince, with a great taste for souls, comes to mind.

That same Prince also created the very first soul gem, so I think it's very probable that the Soul Cairn is somehow tied to him.

Who?

Molag Bal

Molag Bal, King of Strife, is second only to his brother Prince Mehrunes Dagon in the enmity of our Emperor. His lands are the charnel houses the slave pens of Coldharbour, which hold no contrition for those travelers that visit them in error or purpose. That Molag Bal is allowed his holiday at all hearkens back to a treaty of ancient times, when he reputedly lent his infernal power to the creation of the first soulgems.

Well then, it's no coincidence that whole lore with Vampire Lords, Molag Bal, and the Soul Cairn are all intertwined in Dawnguard.

And considering ESO's lore with Bal having Mannimarco as his agent/champion pretty much makes him a patron to necromancers. So he must have some stake in the Soul Carin.

I agree. Probably some kind of "black market" for the Daedra, only Bal gets a "cut" from all of the Ideal Masters' "transactions."

Well, Mankar Camoran was able to create his own pocket realm in Oblivion, so why not others? And then of course the Necromancer in Skyrim turning Azura's Star into his own pocket realm of Oblivion too.

The Ideal Masters remind a little of http://www.supernaturalwiki.com/index.php?title=Crossroads_Demon from Supernatural, at least in the way they trade power for (ultimately) the person's soul. The black market vibe definitely appeals, and it makes sense that their soul trafficking gives them some measure of leverage against the more powerful Daedric Princes.