I've been levelling as a stealthy mage guy. My main skills are sneak, destruction (specialized to shock spells), and illusion(invisibility) with a bit of dabbing in archery. I play on expert and the only problems I have with killing is running out of magicka if there's 5+ guys, and I'm only in a situation like that if I fail to assassinate them all properly, or kill them all at once.
So as a destruction user, I have no problems with it.
And there's my two cents
Exactly. My 56 mage playing on master difficulty has gotten all his kills with destruction spells.
The only problem seen with destruction on the forums is a small group of people who want a seriously gimped, 1 dimensional, badly played character to be 'balanced'.
Just ignore them and have fun.