Visually there is not a whole lot of difference. If you took pictures of all the armor classes in oblivion and ,without looking at the names, sort them up into light and heavy armor. How would you do that? Leather fur and chain mail look light enough. We can drop them into the light pile, but when you get up to mithril, eleven and glass. They look just like the heavy armors. They might literally be lighter, but they don't look like they are.
The next problem I have with them is how different they are in game play. What is the difference between light and heavy armor? Threw all my time playing in oblivion, I only noticed one difference. Light armor weighted less and heaver armor had more armor. Granted that's a difference, but it's not a very distinct difference. It's the same kind of difference between armors in the same grouping. Fur armor weighs less and leather has more armor.
Now that I have defined what I think is the problem, I want to talk about how I think it can be changed. First I say change the names of the classes. Light and heavy are not very descriptive. Make it something like leather and metal. If you see leather armor, you know right away it's leather. If you see metal, you know right away it's metal. Next the two armors should have unique benefits not available to the other armor. You can do this quite easy with perks. For example, Archery perks that let you shoot faster in leather armor, or sword perks that make your charges more deadly when you wear metal armor.
This is just what I think. If you think the system is fine or that it should be handled a different way, please post.