At least in oblivion, only skills that mattered would cause you to level up. At the rate that lockpicking increases in skill, I have to be careful on my warrior not to pick to many locks or else I'll level up prematurely as well.
Speaking of my warrior, at level 23, I run into a bandit marauder with a one hand and shield, with steel armor and me in full orcish, yet he outdamages me by quite a margin, requiring me to drink all my potions just to stay alive long enough.
TL;DR - Skills that aren't important should not cause a level up. It is extremely easy to level, so much so that I have 3 level 20ish characters and I haven't even had the game a week.