Reading the comments on Reddit about this subject, I see I'm not the only one who feels greatly dissatisfied with several of the limitations put on settlements by Bethesda. Here are the top complaints I have (that I see are echoed by many others on FB and Reddit):
1. The settlements having a cap on Charisma+any modifiers you've got going on is really a lousy system. It limits the number of people you can have at any one settlement, regardless of the fact you can build there to your heart's content in some cases. How are you supposed to 'rebuild the Commonwealth' to help push the raiders, greenskins, and ferals out if you can only have 21 people at any one settlement any any time? My ability to become a true Commonwealth Baron is now greatly diminished.
2. People sent out as traders from one settlement to others still take up a bed space at a settlement, despite not being 'in town' most of the time. They're free-range roamers, and they never actually sleep in a bed at a settlement once they've been set up as a trader, so their number shouldn't count as part of the cap on that settlement, really. But they do. Ergh.
3. Some of the settlements allow you to build more than others. Kingsport Lighthouse, Sanctuary, and Abernathy Farm have disproportionately greater build possibilities than Oberland Station, Outpost Zimonja, Coastal Cottage, or Country Crossing, for instance. As a result, the smaller settlements are limited in what you can build there. To get the maximum settlers there (physically put down 21 beds, with the minimum in walls/ceilings, electricity, water, and food), you can't include some of the perks, like trading stalls or scavenger/crafting stations. That lowers the settler's happiness factor with you. It's a rigged system, not in your favor.
4. What the settlers want is not always obvious. When the bed, water, or food situation is bad, they tell you, but everything else is just a load of meaningless complaining. "Haven't had clean fingernails" tells me I should put down a bathtub, but it doesn't mention soap and a dishrag, which are part of that package to get them to stop complaining about dirty fingernails. "Having blisters" tells me to set up a medical stall, but it doesn't necessarily stop that complaint, especially if that stall is a level one (basic) stall, and they need a level three stall to accommodate the community.
5. Assigning settlers to certain tasks -- some of them are resistant to going where you direct them (especially if they like to patrol the grounds rather than work in the fields with the crops). They'll say 'OKAY', and then walk away or never go where you want them to. Not sure if this is an XBOX ONE glitch or a bad software design.
6. Permanent settlers at some settlements take up bed spaces. Like at Warwick Homestead and Somerville Place. It makes sense, I get it, but it also limits how many new settlers I can draw in or send to that settlement -- settlers that will actually defend the place (the kids just run away and you can't arm them with a weapon or put armor on them to protect them) or who can be assigned tasks (again, the kids only work on the crops...only the advlts can be sent to do things, like man the artillery or a trading stall). That limits my organizational skills for those settlements, because now I have to account for useless bodies that take up space.
7. There are lots of other locations around the Commonwealth that would make awesome settlement possibilities, but you can't build there. Like Fairlane Hill Estates, anywhere in Salem, or somewhere in the Boston Waterfront area.
8. Your power armor spot is marked on the map for you to find it, if you left it at a settlement...but companions are not. I often forget which companions I sent to a certain settlement. It sure would be nice for your Pip-Boy to tell you (under WORKSHOPS section), not only how many people are there, but which companions are located there, too.
Would love to see Bethesda address these issues in future DLC releases, as a big part of the draw of this game for me was the settlement buildings and rebuilding the Commonwealth.