My biggest problem with the settlements being in the next TES is, going by what we've seen with Fallout 4, the likelihood that expansions and DLC will be gutted. If you look at FO4 DLC, we've gotten one (Far Harbor) that can be considered a proper expansion, in that new lands, creatures, quests and such are added to the game. Of course, Nuka World has yet to be relased, but based on what I've seen so far, I'm definitely underwhelmed.
Everything else has been little workshop and settlement building packs that seem to do nothing more than unlock content that was already in the vanilla game files, and aimed pretty much exclusively at console players, as PC players already have access to mods that add the same kind of new assets and features to settlements.
I don't want to see the next TES give us Hearthfire spread out over four or five "expansions", and us having to give up another Shivering Isles or Dawnguard in return.