I only explore quest locations, if a quest takes me to a dungeon, I won't leave the place without checking every nook.
Problem with this is, there are SO MANY amazing places in this game that quests will never take you to. Eventually you're going to have to start looking for things on your own.
I personally don't think that you can explore too much. A lot of people are afraid that if they visit a dungeon too soon, and then have to go back to it for a quest, it breaks the game. I've never had this problem with any TES game, and don't expect to. gamesas builds these massive worlds because they want you to explore them.
The only thing you really have to worry about is if you've sold some quest items which will cause you to not complete quests later on.
I'm fairly certain *most* quest items are unsaleable. Only instance I can think of involves a bottle of skooma and a certain mine.