Someone could ask the legitimate question:
"Why bother?"
I am not so much disagreeing with you on the subject, just highlighting a problem that I feel even you adressed here.
Who here remembers the teeny tiny little ants on the tree stumps in Skyrim?
Thats right. They go so far as to animate LITTLE ANTS in Skyrim, something that most gamers wouldn't give 2 hoots about when playing.
If it's obsessive than I am curious at which point would you decide that it's to much?
World building for an open world game is of course one of the key aspects you have to work on. But not everything needs to be detailed up to the point where you can count the ants on a tree just for those 2 players from 200 that might notice it. At least NOT on the expense of story-telling and gameplay. That's really a bit to much in my opinion.
See mine craft builds a very engaging and entertaining world - not for me, I hate the game, but many others love it. And they don't have those Skyrim/Fallout 4 like obsession to detail when it comes to the world. It's literally just all made of crude blocks. But it works. As long we only talk about the attention of detail of course!
Because on the other hand, story telling, NPCs, dialogue, pacing etc. all create tonality and add to the world building as well. It is not some kind of either chose world building or story telling. Particularly today you can do so much more with the tools at your hand where visuals are a part of it, not to mention the understanding is also better. Where developers would once decide on something because it feels right you have a lot of tropes and research to fall back on today, where you simply know what kind of button you have to push in the minds of gamers/movie goes/book readers to create a certain reaction from your audience.
To make it short, World Building is the sum of all parts, its not anymore a situation where they are separated from each other. Otherwise you could create Fallout 4 as a clone of mine craft and just sell it as that but with better visuals - obviously. But how much of an RPG would be left than? So World Building contains graphic, story telling and gameplay. Simply because all of those have an effect on you when you're playing a game. You can leave one of those elements out, sure. But it changes the experience. That's why mine craft for example as game is a one-hit-wonder, while Fallout as franchise is still going even after more than 15 years. Because of the story and setting being a part of its World.
Not to mention that this idea of only the World Building - with your definition, counts is very contradicting to the core of Fallout that was ALWAYS about telling story and the interaction between the Player, NPCs and the (post apocalyptic) world (of Fallout).
It is in that sense interesting to see how the design ideas of the old and new games are clashing together as I do also remember quotes from the original creators where the moral choices and story in the post-apoc world was more important for them than offering a new plasma rifle. Seriously, if all you have is a "World" even if it might look pretty you could end up with something that Destiny was in the begining. And I really don't think that is a good approach for Fallout as franchise.
Anyway. Those are my 5 cents on it.
*Edit, spot on Gizmo, well played