Technically it makes much more sense. If I was comfortable in heavy armor, shouldn't I be MORE comfortable in light armor, but less protected.
More comfortable does not translate into more effectiveness. I might be able to put heavy armor to combat use very effectively, but just because I'm more comfortable in light armor doesn't mean I'll be able to use it in combat any more effectively. For instance, in plate armor I could use my arm guards to deflect an attack but if I tried to do the same with leather I'd likely lose my arm depending on how I deflect the attack.
You fight differently in plate armor compared to leather, and that is what having a separate skill is meant to represent. You know how to use that specific kind of armor in combat. Just because Beth hasn't fleshed out the games to the point that this is actually obvious (and not making the illusion that the skill is passive) does not mean that they should just scrap the systems. Thats just lazy.
You should be able to wear any kind of armor and get a good amount of protection from it, and then add perks that make you better in certain types.
In Morrowin thats precisely how things worked. But because, as it is in real life, if you don't know how to use that specific kind of armor its weak points are going to be easily abused by your opponent if they have any idea what they are doing. (which as far as ES is concerned, is a given) That is where the armor point system comes in. The lower it is, the less effective you are at using your armor. It can never reach zero so long as you're using a fully repaired (or undamaged) piece of armor, but it won't reach its max potential if you don't know what you're doing.
Honestly, instead of removing the skills, they should have spent developing a decent locational damage system like F3's, except taking into account the individual armor pieces and where flesh/cloth/chain is exposed. Having a higher armor skill means these areas that are relatively unarmored (or not resistant to direct attacks anyway) are going to be more easily defended by you. Having a lower one makes it easier for your opponent to abuse them for high damage.
But to PRG purists, it's one less skill to grind, so it "svcks"
The only time I ever truly worried about grining skills or attributes was when I either A) decided to play on the hardest difficulty without abusing anything or B) wanted a specific level in an attribute/skill that I wouldn't achieve through normal gameplay. (like if I wanted to start the game as a master swordsman from another land for instance, I'd grind my Long Blade skill then go on to roleplay afterwards)
If an RPG is forcing you to focus on grinding skills outside of whats natural (early game thats pretty much what you're supposed to do, as it is in any RPG) then the mechanics have been thought out very well or for some reason were made that way to cater to people who like number crunching. Oblivion obviously did the former but if you're defining ES games by what Oblivion has done then you're doing it wrong.