Definitely join a Faction/Guild/house that fits your build.
It's a great way to slowly understand what is expected of you, and get some easy experience/gold.
I'd need to see your entire build to know more, but the Fighter's Guild, House Redoran, and the Imperial Legion are made for combat orientated builds.
Doing the main quest alongside one of these is also good.
I don't recommend random exploration of every cave you see, yet. You will likely stumble across enemies too tough for you.
Keep in mind that Morrowind starts much slower, and the first 10 levels can be especially difficult and frustrating.