I found destruction ok in the early levels but something really strange happened around L38 in the level scaling that I don't understand with the build I used. I was getting one shotted by typical bears all of a sudden just as an example no matter what i did. That game became unplayable.
But that doesn't have anything to do with Destruction, it's because (I assume) you were wearing no armor and had almost no health. That's a completely separate issue from Destruction. You can use Destruction just fine while wearing heavy armor or even holding a shield, and you can put more points into Health. If people's main complaint is that they die too quickly in melee, I really don't think that has anything to do with Destruction.