One building being razed wouldn't really affect the performance at all.
Your right because that would be a scripted event.
The only reason I see not to have fully destructable environments would be if it takes way too much time for Bethesda, which I doubt
I really get tired of saying this but here we go again anyway.
Bethesda Open game world + fully destructable enviroment + 32bit runtime =
NOT ENOUGH MEMORY The sooner you people get this through your head the better off we will all be. You can't add ANYTHING as resource intensive and not give up the game world. You can't have both and most of us don't want to give up our game world to get features that can be experienced in other games.