I have to agree. The Settlement building is my personal biggest gripe with the game (other than the crashing in Boston).
I'm currently walling off the Starlight Drive-In (Great Settlement choice btw), and I did not know that Raiders could take my Power Armour. I'll be taking those Fusion Cores out when I next get on. I have 2 Power Armour sets in Sanctuary Hills and 2 at the Starlight Drive-In.
As for Defences, I've been putting Turrets in elevated positions, but when Abernathy Farm was attacked (by 8 Super Mutants and some Mutant Hounds no less), the Abernathys were ill-equipped to fight them off. Within 15 seconds I had to gun down 4 Mutants with my Combat Shotgun and then sprint to save the family. The Mutants had perfect accuracy with their Molotov spamming on my Turrets (damaged 2 of them to the point of needing repairs), and Lucy ate one to the face. I saved the family and started building walls (even though I saw that the Mutants spawned inside the southern field). I had 4 HMG Turrets up, 2 on the house and 2 at the settlement southern perimeter on top of Premade Small Metal Shacks. The north was virtually undefended.
Those spawn points really need reworking if we are to actually defend against attacks. Also, putting Guard Towers and Turrets on top of buildings really helps. I also never fast travel, though I do have about 17 spare Fusion Cores (not including those inside my Power Armour sets). I pretty much had to run from Parsons State Insane Asylum to the Abernathy Farm to save them. I got harassed a bit, including by a Mirelurk Hunter, but I just ignored most distractions and carried on running.
Also, do I ever get the chance to build better quality settlements? I mean, Junk Fences and Wood/Metal Shacks are fine at the start of the game, but I really wanted to build something more... opulent. Concrete walls, electrified fences, biometric security gates, automated defence protocols, designated safe zones and panic rooms for settlers and some control over their AI to assign who should avoid combat and who should fight and what weapons/armour everyone uses (rather than just placing things in someone's inventory and "hoping" they'll use it).
All in all, it's kind of disappointing. Weren't they working on Fallout 4 for seven years (since '08)? This settlement building needs another 2 years of redesigning and memory fixes to allow for a more robust system to be created. At the moment, it feels little more than lip service, a kind of forced concession Bethesda made to placate fans.
Oh, and one final tip; the first Wooden Railing in the Misc. Wood Structures is great for improvised low cover on high rooftops and the like. I even used file cabinets to elevate some turrets and protect the generators at the same time. You really need to be creative to make a more self-sufficient defence system given the semi-botched tools we've been given.
And yes, I love strategy games, so I LIVE for this kind of thing! I wish I had more CONTROL!