I turned off the HUD. It totally crazy. I hated the immersion of Skyrim and the quests and all until I turned off the HUD. I don't know it just feel more real....I feel like I am exploring and must be aware of danger. People who want immersion should try it.
Of course the no crosshair with no indiction of what you pick up is totally stupid but you have to get past over that..
PC users can turn off the quest markers and/or the compass entirely. Sadly, no option to turn off location markers while leaving the compass itself untouched, though. It's how I've been playing since day 1.