Its happened to a lot of people, and it does seem to be a bug.
As in your case, there is no warning of an attack, and no signs of any attack in progress. You can either fast-travel in to the settlement, or walk in, the trigger seems to be the same. As you stand there, everything destructs around you. The settlers don't react, and don't behave as if they are being attacked.
When one of my biggest and most expensive settlements was affected, I tried to bypass the bug by discarding about three hours of gameplay and going back to a really old save. I then avoided that settlement for many in-game days (I actually levelled up three times before I went back!). I obsessively checked the pip-boy for attack warnings, but there were none. It made no difference at all: the settlement was fine until the moment I finally arrived - then it destructed anyway.....