I have played nearly 300 hours and many characters. My take on some of the skill trees differs from some of your own, OP.
In a general sense, naming the biggest flaw in Skyrim’s perks system is a tossup for me.
First it sets an expectation that the higher a perk is on the tree the more useful it will be. This is not always so because many of the top end abilities are low percentage based occurrences and some of the mid-range level abilities seem to be less than useful filler.
Secondly in order to flesh out the perk trees for skills you actually like to use, often times you will have to level up a skill you do not enjoy or ever want to make regular use of. It can make playing feel like a chore.
Third, the synergy between crafting skills can create exploit grade abuses of nearly any skill or game ability. I would contend that these skills should be rethought for better play.
So, here is my take on each skill tree.
One Handed: Most useful attack skill due to versatility. You can use a shield, spell, or second weapon in the off hand. Dual weapon wield was “most” useful for sneak based melee characters.
Two Handed: Slow with a big hit. You’ll need to use cover mid-levels to get close to mage like enemies in mini packs.
Block: Very nice advantages on the left side of the tree and the top ability. Disarming bash sounds nice until you realize it is only effective 10 -20% of the time. Power bash viability is linked to your stamina as it will be shared with your weapon skill…
Heavy Armor: Most useful for less staminia reliant and non-shield using builds and takes the a couple more perks to fully make use of. You can easily hit armor cap with either armor skill.
Light Armor: Most useful for stamina intensive or sneak use intensive builds. You can easily hit armor cap with either armor skill.
Archery: Well balanced on its own. With sneak you’re a very effective one shot sniper. Add in crafting and you can become an invisible, long range, death canon.
Lockpicking: Five points here can be extremely useful “IF” you do not want to rely heavily on selling items to merchants for gold or crafting for enchanted armor or weapons and would rather find them. Otherwise, it’s a questionable perk tree to invest in.
Sneak: Frustrating early on but becomes incredibly useful at the high end with some choice perks. Its play enjoyment depends on your degree of patience.
Pickpocketing: It’s hard to justify spending perks points here but even without them this skill can be incredibly useful for what amounts to free training in most any skill up to 50 points. Most all NPCs do not carry much coin. They have the occasional valuable you can nab to sell for coin if you have a fence.
Speech: Perhaps the game’s least useful skill. Maybe if you just want to RP a wealthy merchant or noble but it has so little actual game use that it can only be looked at as a waste. Levels easily by way of training or natural use in selling “tons” of items… Item value seems to have some effect. You might could make it OP with some enchantig. *cough*

Alteration: A high value perk tree with good returns. The spells are very useful from low level to high. Paralysis often seems more combat useful than most destruction spells. Add in damage reduction…
Conjuration: Very cool in concept but only mediocre in play until you top end the perks. In that sense it might be the most balanced tree in the game. lol
Destruction: The least punch of any combat skill and feels like support ability rather than a primary means of offense. It’s more of a mid-range softner with some AOE potential. However, AOE is of limited value as there are rarely big mobs.
Restoration: It’s the one school where I really feel the perks can be more useful than the actual spells.
Illusion: The most perk intensive skill and tree in the game to maintain usefulness for while leveling. For the cost, it can be fun and allow some cool gimmick options but it’s more practical and perk cheaper to just arrow an opponent in the knee. Take dual casting…
Alchemy: Frustrating to level and useless early game. You can buy or find better. However, it becomes very useful at the top end of the skill value and perk tree. A long term investment with big payoffs.
Enchantment: Simple to use and level. It becomes inordinately powerful and useful to any character with time. Perhaps the best perk investment to return value tree in the game.
Smithing: A must for any character relying on physical weapons and armor. Even if you don’t want to craft your own gear its valuable for improving what you have. Easy and cheap to level.