So I walked through a red brick door and am somewhere else with only a solid wall behind me. Does this mean I need to explore and find another teleporter that will get me back to where I had already been?

Not necessarily. But it does means you have to go and explore.

A few of the dungeons don't seem to have a way to get back when you get teleported. Exploring is key. If there's no way back, magic is often the key. I did find one dungeon where I had already found the exit of the teleport by the time I tried it. Made things simple.

And some red brick doors don't work. So good luck telling if you missed a lever or wheel or just found a non working red brick door!

It's hard when you find a room with six switches, and you're not supposed to flip them all.

Well I found my way back out again. Fun dungeon! I just explored and found more teleporters that brought me back to the dungeon entrance. Also I found way more treasure in that dungeon than any dungeon yet; there were at least 12 piles of loot in a couple rooms. In this dungeon, every red brick wall activated upon clicking it/walking through it, but next time I may not be so lucky.