It'll be tough to ignore combat because you
will be attacked. But until you know what you're doing, don't attack NPC's that aren't hostile.
There aren't many cues as to what to do like in the newer games, so wander, talk to people but be careful and save often. It's very easy to run into a situation where even a high level character would have trouble.
There are several guilds/factions to join, so investigate them as a means to advance your character.
You gain experience by using skills...ie, you'll gain experience in the longsword skill by using a longsword or the security skill by picking locks. Only major and minor skill increases will lead to a level up and it takes 10 of those increases to do so.
There's a lot of other stuff, but if you're careful, you can find good advice on the http://www.uesp.net/wiki/Main_Page or the ES http://elderscrolls.wikia.com/wiki/The_Elder_Scrolls_Wiki. Personally, I prefer the UESP, but that may be because I've been using it so long.