I've been wandering aimlessly around this barren map, and I keep finding tiny settlements, with people living in what are essentially shanties, while towns with actual buildings are just pointlessly sitting. Will people start living in them if I waste the time to clear them of enemies?

As of now, most of what populates the map seems to have no purpose other than just taking up space.

Those urban areas of the Commonwealth are basically equal to that of the DC ruins. A warzone that belongs to the likes of raiders, super mutants, wandering feral ghouls, and other such life forms of the waste.

I've noticed that it doesn't seem to matter how many you kill, as they keep spawning back. At least, that seems to be the case for the Cambridge/Charlestown area. Concord seems to be empty in general besides the scripted main quest moment with the raiders. Haven't been in Lexington for a while to confirm anything.

Those are too big of targets to just start settling now. Diamond City is already established but trying to build something like that in Concord wouldn't last long. Too much hostile traffic

The only places that can be cleared out and settled (and the player has to do all the work to set it up) are the ones that have a little farm looking icon on the map.

Actually many places with a settlement workshop bench can be turned into a settlement, just build a recruitment beacon, power it and watch settlers arrive, with only a few rare exceptions.

After some time there are raiders respawning in Concord. At least in my game.

But, but.. That is what post-apocalyptic setting is all about right? The desolation, walking around ruins alone. That mood is what makes me love this games. I feel 4 is way to crowded already, they have constant war between factions, and live in a wasteland, yet they manage to somehow reproduce at such high rate

Most of them have that farm icon though I forgot there are a few that don't.