Another problem is that you don't even move faster when unarmored, so bringing up mobility is really a null point. In fact, light armor with wind walker perk gives you increased mobility over someone who's unarmored. But really, even if sprinting cost less stamina with unarmored, it'd still be pretty bad 'cause the speed that matters most by far is the default run speed. If unarmored were 100%, light armor 90%, heavy 80% or something like that, unarmored could actually be seen as worth consideration for archers and mages.
I was thinking about this last night. There really should be an Unarmoured skill with its own perks, I think. There should also be a few changes made to Light Armour.
I think the Unarmoured skill tree should have, as its first perk, a 5-ranked "Spell damage/effectiveness is increased by 20/40/60/80/100% when wearing no armour" perk. It could also have different branches for different specifications of Unarmoured - one for magic resistance, one for extra movement speed, one perhaps for teleportation while sneaking, one for extra glass cannon damage (each 3-ranked, like the weapon-specific perks in 1-hand and 2-hand weapons).
Light armour's first perk shouldn't be "Light armour affords 20-100% more protection." It should be "Chance to dodge attacks is increased by 20/25/30/35/40%." There could be a 3-ranked perk elsewhere in the tree for damage resistance.
Just a few thoughts, which clearly need refinement. Food for thought/potential mods, perhaps?
EDIT: I also forgot to mention that I also believe the heaviness of armour should affect your spells' effectiveness. Robes (not necessarily all clothing) should have an innate +20% spell effectiveness bonus; Light armour should have normal spell effectiveness; Heavy armour should have -20%. Otherwise there's no point in wearing anything other than Heavy Armour as a mage.