(I'm bumping this thread, so as not to start a new thread on a topic which already has one.)
I'm not high level--only 26--and so I have a ways to go before I can't do the pure Destruction route anymore. Fortunately, I'm taking the time to upgrade Archery

However, one thing that's somewhat irksome is how, in the current system, upgrades to Destruction's damage come in 5 "chunks," plus two more "chunks" for the 2 perk point magic school damage talents.
This means that when you're nearing the next tier of Destruction spell, things become harder to kill. Now this isn't a HUGE problem, as you can just pace yourself and do easier content until you hit the next spell tier, but it is an annoyance.
Yes, weapon wielders also have discrete damage upgrades in the form of new weapons, but aside from this they also have boosts in damage from increasing their weapons' skill levels and from a larger number of perks that increase weapon damage.