Why.
By the same rules, if I choose to be a merchant, max out my speech, then I should also have the option to talk the dragons into settling their differences over a game of dragon scrabble. The most infamous and most dangerous board game in the world.
Just imagine. A game where you spell a word and the room bursts into flame.
As a unrelated issue, did anyone else chuckle at the loading screen quote which stated that fights between dragons are in fact, just a heated debate?
Okay, but enough of that. Back to basics.
Get over your ego. Just that.. take a deep breath, and realise that difficulty is relative, and your innate need to beat a game on its hardest difficulty is, while a worthy goal, is getting in the way of you enjoying the game as it was meant to be. Let go of the ego, and the fun will begin.
Loved your bit about the dragon scrabble. Hilarious!

As for your other comments, though: I would say that while difficulty is relative, and is largely based upon the choices the player makes and the player's overall skill, balance should also be considered relative to each encounter. For instance, if there were lots of opportunities for Speechcraft to give you an edge in combat by convincing someone to help you, changing the way the combat plays out, etc. then I would say it would be balanced despite making you weaker overall in terms of level scaling. In the same way, if Destruction magic allowed me to sometimes shine more than a warrior under certain challenges, I would also say it would really differentiate the playstyles and make it feel more than just "dealing damage".
While it's true that there are some monsters that are resistant to physical damage, and have weaknesses to magic, I really don't get that feeling like "Oh, this is where being a Destruction mage makes things interesting!". Never am I swarmed by enough enemies to warrant unleashing a ton of my AoEs, especially at the risk of killing/pissing off my companion. I of course still have lots of fun with my mage, but I secretly wish there were more challenges tailored to Destruction magic's strengths. I would have loved to have had more variety, like spells that could cripple the performance of enemies... Alas, I miss a lot of the old spell effects of yore.
Ultimately, I want to feel like any combination is viable (at the same difficulty) and unique, all while leading to a different set of obstacles. Skyrim does have this to a degree, but I feel like some more work could go into tweaking things like perks and the skills themselves. I guess I'm just the type who obsesses over balance to the point where it may break my immersion if I find out that many combinations are outclassed by others. In the past, what I did was just download mods, so I don't see this being any different. I'd take downloading a mod any day as opposed to tweaking the overall difficulty. The reason is that why I may want to be stronger (or at least have more interesting options) in Destruction, I don't want to take less damage from a lower difficulty, or have my companions kill faster. It's just one teeny, tiny aspect that I would like to change, and I feel that simply sliding the difficult up or down affects more than what I'd like.