The leveling system.
Say what you will about XP based games, but I've always found TES relied too much on plotting out your skill gains to get those precious x5s. It kills my sense of immersion far, far more than any other problem the series might have.
This. TES system is interesting, but it just isn't practical. Monsters leveled with the character just make it worse. No matter where you are in the game, if you level up non-combat related abilities, the game doesn't care. You leveled up. As a result, the game just gave you more difficult monsters to fight. Now my mage that has skills in alchemy, mysticisim, illusion, and alteration can't possibly keep up with the monsters in the world.
Forget taking something from Fallout, which the entire S.P.E.C.I.A.L. system, TAG skills and traits is nice, I really enjoy Bioware's skill trees myself. The kinds you find in Knights of the Old republic and Dragon Age, etc.