Is the soul in your gem actually a soul or a life force kind of thing?
people who were trapped in a Soul Gem face http://www.uesp.net/wiki/Lore:Soul_Cairn as their afterlife (as shown in the "Dawnguard" main quest).
EDIT: from the uesp page:
As funnybunny said. Your soul goes to the Soul Cairn.