I play with my compass off and as little use of the overhead map as possible. I open up the map, look at where the location is in relation to other places, and then don't look at it again until I have to go somewhere else. Navigation using the sun and stars.
In SkyrimPrefs.ini ...
under [Interface]
bShowCompass=0
That's how I have it.
Just to be clear, it's the SkyrimPrefs.ini file under Documents\My Games\Skyrim\ (Windows 7, XP is slightly different), not the one under the install directory (Skyrim\Skyrim).