Well first of all congratulations for a well thought out essay (yes you can call it that, definitely), that didn′t seem like a wall of text at all.
I never played Morrowind, didn′t like Obilivion, but currently find myself enjoying Skyrim quite a lot. My previous RPG history includes The Fallout series ("Tactics and BoS not included..) from the beginning and then some random rpg:s here and there.
Now, even though I have 0 experience in Morrowind, you were able to sum up the differences between it and newer installments and I have to say, that in some pseudo-level, I do agree.
What it′s all about of course, is money. Studios like Bethesda (maybe all of them?) are VERY dependant on each and every one of their publications in the way, that one mis-step can well run down the whole company. This is the same with "post-big five era" Hollywood film studios, that after losing their chance (end of top-down monopoly = production, distribution and exhibition) to gain revenue from distribution and exhibition, were forced to produce those BIG HOLLYWOOD FILMS we all know and hate, starting with Jaws, with the Transformers kind of movies as a more current example. This is because after losing a big portion of their chance for revenue, every movie had to make it in the box-office, or else the whole studio would fail. The birth of TV also played a part, but this is not a topic on the history of the cinema..

So anyway, game studios are a plenty nowadays, which means fierce competition, which doesn′t mean better games, but more popular ones, which to some (like you for example) means the death of "real games". Less and less is left to the effort and intelligence of the player in order to avoid the game seeming like a mess, resulting in the types of issues you talk about in your post. It is a sad state of affairs among our beloved form of media, but it is not something that will likely take a different road anytime soon, although there will always be exceptions from the brave.
Skyrim is an exceptional game, far better than most, and it is a sandbox game, which gives us as players atleast some control over our gaming experiences. I′d advice you to pick the best you can (judging from the end of your post you have done this already

) and wait for those rare gems of games that sometimes appear. The tricky thing is though, that once those games become popular, they get more money, more investors, more fierce boardroom meetings and finally end up exactly where Bethesda is heading.
But then there will always, hopefully, be a new studio, and a a new idea...