So I was testing this... http://i42.tinypic.com/ds7m8.jpg
I know the game wasn't going to be infinitely large; but I expected just a seemingly endless expanse of ocean in the North and East, and impassable mountains to the South and West. As you can see in the image, there are many little islands out in the direction that "I cannot go" to.
Although, I am still technically out of the map (I am literally under the clouds at the edges of the map) and there was actually a dungeon a little ways south of where I took the image. Which I thought was bad ass.