Most games give exp, you level up and do what you like. TES makes you use skills to raise them, so even if you can take any perk, it's still better than most. As for mages taking health, would a hedge wizard, or a witch living in the heart of a forest, not have more health and less magicka than a guild mage living in a city?
Why would that be necessarily the case? Maybe they'd have less because they'd have too rough a life if we are getting realistic here. I'm not really talking about "hyper-realism" though. Said hedge-wizard would be able to have more health in the TES video game universe because they would use other skills besides casting spells anyway. Being in the dangerous woods all the time, its likely they would not be able to purely study magicka like a mage in a nice guild setting, but would frequently have to resort to other aspects of combat to survive.
For the sake of example, these numbers and exact relationships are not necessarily ideal /end disclaimer
For each point you increase each of the following skills, it adds to the total of the baseline increase of the stat increases:
Heavy Armor - 2% per point per level increase to Health and Stamina options. 0% increase to Magicka.
Destruction - 3% per point per level increase in Magicka option. 1% per point per level increase in Stamina option. 0% increase in Health option.
Blade - 3% per point per level increase to Stamina option. 1% per point per level increase to Health option. 0% increase to Magicka.
So if you used all Heavy armor increases, you would gain more by selecting Stamina or Health than you would by selecting Magicka. If you used all Destruction, you would gain more by selecting Magicka than you would by selecting Stamina, but earn no bonus if you selected Health. This way, the amount of increase is logical to the type of activity you performed.