Alright, here are my thoughts:
Destruction: Of course Destruction is underpowered, I mean come on! You get RIDICULOUS spells like Blizzard and Fire Storm.
Conjuration: Isn't a good tree to compare with other trees. Mostly I consider it as a "distract the enemy" kind of magic.
Illusion: Similar to Conjuration. Used to distract the enemy or stop yourself getting swarmed by them.
Smithing, Enchanting, Alchemy: Do agree the crafting skills are a little OP. But mainly just levels up too fast.
Lockpicking: It's definitely worthless with the Skeleton Key, but you lose it if you want to finish the Thieves Guild questline. My opinion is that it levels up way to slow after level 50.
Archery: Overpowered but it's designed to be that way because most enemies will run at you to fight, which means switching to melee so you do more damage and take less hits overall.
One-handed: Dual wielding 1 handed weapons is definitely [censored] in blocking, but that's spread evenly with armor. You want to do more damage? You need good armor.
Speech: If you're having way too many items to sell and the merchants aren't rich enough, the later perks are quite good. Plus, it's basically a cheapskate's way out of not wasting gold on bribes (like me).
So most of the OP skills are connected to the UP skills which evens them out. Conclusion: Perk system isn't THAT flawed/bad.