Have you tried using your fists?
In "Oblivion" many people complained there was no "Unarmoured" skill, seemingly completely oblivious to the fact that an unarmoured character using hand to hand naturally moved faster than those lumbered with gear so there was no reason to create an "skill" to artificially enhance it.
Azrael
The Nord with the Sword
Except for the fact that when you wore armor you gained skill level when you got hit, increasing your defense while wearing that armor type. Without an unarmored skill, your armor was always 0 when you wore just clothing.
Same thing without a H2H skill in Skyrim: you can use your fists if you want, but the only way you can increase damage with them is to either make a khajiit or wear heavy armor gloves and get the pugilist enchantment. What if I wanted to be an orc monk? Monks don't wear big, bulky gauntlets, and I'm pretty sure that the type of monk who trains in martial arts also knows how to take a hit a lot better than the average person.
The removal of acrobatics and athletics is a huge loss, too. I don't expect an old mage to run at nearly the same speed as a hunter in his prime, and I
definitely don't expect to see a nord in big, bulky armor jump as high as a khajiit in his birthday suit.