You are not doing anything wrong, that is an occasional glitch? Are you on PC or console (Xbox, PS3)?
From my old notes from the UOP readme, I think:
BaPicking up the Sigil Stone in an Oblivion plane doesn't do what it's supposed to: the player stays put, the area fills with white fog, and the gate doesn't close.
Solution: For some unknown reason the Oblivion plane you're in loses its hook to the gate in Tamriel you used, best solution is if you are afflicted with this bug is to open your console and type in coc AnvilBay and you'll end up in Anvil's bay. Also this may be avoidable if you never reload while inside Oblivion planes and then go back into the same gate. If you have to reload while in an Oblivion gate quit the game entirely and restart it to ensure that memory is properly flushed out.