We need H2H as a skill back.
What would you include in the Unarmed skill tree?
I'm thinking the first perk should be one that adds half your Unarmed skill level to your Unarmed attack damage, as long as you aren't wearing armored gloves (you sacrifice the +25% bonus to defense from all-light/all-heavy armor suits, and a little defense from the gloves themself, in exchange for much higher damage).
After that it would branch into "mystic" and "street fighter" unarmed styles. The mystic would basically combine unarmed attacks and magic use, casting spells with his hands: chance to paralzye on strike, stamina damage on strike, and an automatic low-level Reflect Damage type spell any time you have both hands empty. The street fighter would specialize in physical manipulation of his opponent: power attack has a chance to throw an enemy, chance to disarm, high chance to stagger, chance to "break limb" (combat penalties applied).