Whenever I was browsing the Skyrim forums, I would always sigh at people who whined at the simplified path the game seemed to be taking. I would dismiss them as butthurt hardcoe RPG fans, and move on.

But after watching a video of Daggerfall character creation, and with seeing how far it came from Daggerfall to the removal of classes, birthsigns, and attributes in Skyrim, I must say I agree that the game has been scaled down an insane amount.

So, it got me wondering, why is it like this?

The short form explanation is marketing and demographics. Develop a product to the broadest margin to increase profit (games have been developed for 80% console, 20% PC or thereabouts since day two).

Technology information has also compressed time for many people and they expect more bang for their buck in as little time as possible.

BGS is an entertainment business now and not an indy studio, and they want to be as profitable as possible for their shareholders. Good business sense IMO.

People want instant gratification. They immediately want to pick up a weapon and slay dragons.

Look at the difference between the Daggerfall and Morrowind character creation. Morrowinds creation looks pale when you also look at the one of Daggerfall. Between Morrowind and Oblivion didn't change much, only the decrease from 10 to 7 class skills but we lost a few skills in the process (Spear, axe, medium armor, enchant, unarmored on the top of my head). In Skyrim, we don't have any major skill at all, just the skills. With attributes bundled in three main variables: health, stamina and magicka. It does make sense, as some attributes influenced those variables. IMO, it went from this:
Level up, please choose 3 out of 10 attributes-->Attributes-->Increase in Health, Magicka and Fatigue
to this:
Level up please choose one (or more)--> Increase in Health, Magicka and Stamina
I must admit, some things are lost (how are we going to calculate running speed, disposition, encumbrance etc. without attributes?), but I think that Bethesda will figure something out. I really hope that the perks are good, and to have a nice look. Not the list of perks like in Fallout, but a real perk tree would be great, for me that is.

Because Bethesda Game Studios believes that it will make the game better.

Agree or disagree with the quality of the outcome, but that's the basic reason.

Not in my opinion. Because what you end up with eventually is like Bud Light. Filtered to death so that no one is likely to take major offense, but without any distinct character at all. You can afford that maybe if you have nearly cornered the market, but not when you have major competition at hand. And when, say, TES VI appears right next to, say, Witcher III, there needs to be a distinct reason why one should pick up TES VI and not the Witcher III. Marketing is not the least about brands. The concept of brands is that to some degree, you know even without using it what you will get. That's why brand consistency is a major marketing point. If you produce rich, dark beers all the time and have a reputation for it and suddenly you decide "Nah, screw it all, let's compete with Bud Light", customers will be confused, if not appalled. If you want to do such a thing, it is better to do it under a different brand - or under a derived brand that makes the distinction clear. Otherwise, while you might win some new customers, the loss of long-term faithful customers will more than make that up. As winning new customers is usually much more expensive than keeping existing customers loyal (the latter know the brand, like it and don't need much convincing to buy it), the first order of business should almost always be to ensure that the faithful rally round the flag to provide you with a solid base of customers with a proven re-buy probability. Those new customers that you gain haven't shown yet that they will buy the next game, too. So secure the foundation, and on that foundation, build measures to gain new customers.

Marketing isn't as simple as "What type of games appeals to the largest number of people?" because every company can play that game but the gamers have a finite budget. So you have to provide real reasons why people should buy YOUR product and not the other company's. You won't do that by simply aiming at the most widespread factors. All you'll achieve with such a strategy is an "a dime a dozen" game which is suitable for straight to bargain bin. If you want 50+ bucks you have to communicate specific values you provide. And for the individual customer, the fact that it appeals to the masses is not a reason to buy.