Nope. That's actually been one of the complaints that I've seen made against the game, that settlement building is so optional and unintrusive and it's affects on the game are largely inconsequential
which is funny since you can make Bank and some best weapons from it lol wandering around basically hope you get lucky that you can take out legendary baddies with subpar weapons/armour
This, its optional but an nice bonuses, much like most quests are optional.
I hope there are nice benefits to them, because I plan to spend a lot of time building them. I love when games give you the option to build/personalize your own space or homes. I played Skyrim on the console, and honestly Hearthfire was my favorite DLC for that game. I can't wait to give settlement building a try in FO4.
I might not bother at all with it. The idea of dealing with waves of enemies to defend your settlement is awesome, though.