That's not unrealistic, that's your character's history. You don't start the game as an infant. You start it as someone who's been living there for years, working, adventuring, etc. Classes, major and minor skills, are all a reflection of this history. There's definitely some problems with TES character development/creation, but the existence of classes isn't one of them.
There was a lot lost in the transition between Daggerfall and Morrowind. There's no question about that. But what Morrowind did was craft a large, interesting, intimate, handcrafted experience. It excelled at this. Oblivion, despite what some would claim, is Morrowind 2.0. The design appraoch is almost identical. But they utterly ruined the illusion through stupid decisions, while simultaneously removing other features just as they did between Daggerfall and Morrowind.
It would be totally silly to suggest that these changes were at all on a similar scale.