I just got out of the soul cairn and forgot to go get my soul gem, seems I'm regenerating magic again and I don't see any spell effects active, does it only really matter inside the soul cairn?

I think you're okay as long as the Ideal Masters don't get their hands on it.

Ha they don't move much so I think it's safe...I'm assuming that getting my soul back doesn't really matter outside the Soul Cairn, I'll go back for it later, Bethesda's idea of choice and consequence at it's best