How exactly does this work?

I'm wanting to stay a "pure" mage if I can and only wear robes. I know most posts say to enchant with Fortify Destruction to drop the cost to zero.

I remember one post (but couldn't find it today) talking about getting the Mage Armor perks (3/3) but it appears that only works with Oakflesh/Ebonyskin etc. Am I misreading that? I thought there was something about robes then working the same as armor...but the perk text doesn't support that. Before I take points there I'd like to get a better understanding of the perk itself.

Thanks.

All the spells increase in potency. The only flesh spell not increased in potency is the master spell, Dragonhide, it doesn't increase in power as it already gives you the max damage mitigation - 80%, roughly 567 armour rating.

The other spells increase in potency with the Mage armour perk. You can also increase the duration of the spells with another perk in the tree, Stability. Robes on their own have no armour rating whatsoever, using the spells gives you the armour rating.

Morrowind had an Unarmored skill, where your armor rating went up if you didn't wear armor, Skyrim does not. The alteration perk "Mage Armor" increases the armor rating the spells give you. For it to take effect you'll have to use the spells. Without the spells you'll still have zero armor rating.