AI, quest design, story writing, animations, choices&consequences.

Quests and events that change the world somehow.
Example: After you secure the traderoute leading to town, merchants get new items to sell.
That kind of stuff.

Animations!

there needs to be cloth and hair physics.

I disagree on the need for hair physics - It's an area where Reality tends to be "unrealistic" - at least in my experience. Everyone I know who's hair is less than shoulder-length is quite rigid.

The world simulation. It's always well below the scope of the game. The busy roads, the adventurers like you that you find doing your thing deep in the dungeons. The merchants who actually close their shops and travel to other cities and buy new stocks and then transport them by carriage or by boat. The rich quarters of cities, the shady ones. The BIG cities with busy streets, large buildings you can climb, gamblers, nightlife, prostitution, freelance assassins doing their thing at night. The big crowds during celebrations/ceremonials/large scale conflicts. The seasons changing the everyday life of people and creatures in a visible way. The economy that's sensible to the events. The world being responsive to your actions, to your disguise, to the life and death of their neighbors. The ships traveling from one harbor to another in real time.

First, you've only gone after one thing I've said and falsely insinuated that I have claimed, with red herring (veering off-topic and relating the issue to unrelated examples in an attempt to divert proper attention), that whatever this "reputation system" is is what solely is a determining factor. You've also, with that red herring, disregarded the proper context of what we're discussing and that is a trend of reducing RPG mechanics with successive iteration of this series.

Second, that's not true. Those games aren't wide open and so they don't need to utilize a system similar to, say, Daggerfall, but they had reputation systems. Since said games (let's just go with labeling them as BioWare RPGs in general) are in a more controlled, story-based environment, they handle reputation a bit differently. People do react to what you do, people do recognize you for the type of character you play as, and your choices affect external thoughts on your character. Take KotOR, for example, as it is the freshest in my mind of the BioWare RPGs. When you provide the serum for the Rakghoul disease and risk your life to administer it to infected outcasts, the village very well knows, respects, and admires you for your choice. It is recognized, you gain light side points for it, and your step towards being a purer jedi is very real and reflected in the game. If you join the dueling arena on Taris and rise to fame, people will recognize you for it. If you choose to kindly assist your companions in their issues in any BioWare game, they take note, hence you have a reputation.

In a BioWare RPG, your choices in interacting with the world, the characters, and the story have very real consequences, both good and bad, and you are recognized for what you have done, hence a reputation system. In Skyrim, you can slay the prophetic dragon god of time who wants to devour the world and people don't care. You can become champion of a certain "faction" (again, handful of meaningless quests) and people won't know or care. It has no effect on anything... not in the way the world perceives you, people talk to you, or quests become resolved. An open game like Skyrim needs a different type of reputation system to better manage the variables, but BioWare RPGs certainly have reputation systems. They certainly have people responding to you, choices meaning something, and you see a clear effect of your actions on the world and your character's own image.

It is lunacy to outright disregard the fact of complete and unconditional removal of RPG elements and try and claim they weren't actually removed... or that, in your case, RPG elements weren't RPG elements in the first place and that RPG mechanics have not been removed from previous games... in which case anything that's been removed is, by extension, in your opinion not an RPG element and this whole argument cannot properly exist for there is no proper ground on which you can base an opposing stance. I would very much like to see, in detail not in vague, unsupported claims, reasoning for why Skyrim is somehow not less of an RPG than its predecessors. Compare Skyrim to the pinnacle of RPG mechanics in the series, Daggerfall, and tell me how Skyrim manages to go toe-to-toe with Daggerfall on a purely RPG mechanic level.

I believe the TES games are generally too different to specify anything but...

Bugfixing, better animations, more spoken remarks (not like: "blablabla mudcrabs" or "arrow to the knee") and more compelling storytelling.

World detail and functionality. Character progression. Definition of character through actions - I don't need the game to tell me what I did - I know what I did. Others do respond to me and my behaviors, some more dramatically than others. The Thalmor put a hit out on me!

I've managed to define myself MUCH better in Skyrim than Daggerfall.