I've played Morrowind, Oblivion, and Skyrim. I hated them all at first, but after playing Skyrim extensively I went back and enjoyed Oblivion, never got a chance to replay Morrowind unfortunately. I think I'd have to say Skyrim is my favorite.
To me, every Elder Scrolls should have a unique setting. Morrowind is definitely different than Cyrodiil and Skyrim, and Skyrim and Cyrodiil even share their own set of differences. I preferred Skyrim over the other two, simply because I'm into the Nordic culture, as well as anything related to Vikings and a land of 'ive and fire'; in other words, Skyrim was the perfect setting to me - although Morrowind's mysteriousness and Cyrodiil's familiarity were pretty good.
I also think every Elder Scrolls should have a wide assortment of armors, clothing, robes, weapons, and spells to further enhance the idea of having a 'custom character', and all three games have done a good job in that category. A bit disappointed that spears were in Morrowind and not the other games though.
I also preferred Oblivion and Morrowinds idea of having attributes like Strength, Intelligence, and Agility. They seemed to further define the character. I also like how you unlocked perks along the way rather than choosing them. It just felt better! But Skyrim does a good job of simplifying some of the skills (like doing away with blunt, blade, etc and just saying one-handed and two-handed), but they should not have removed things like athletics or acrobatics.
And of course, all Elder Scrolls games needs a good main quest, fun Daedric quests, a warrior's questline, a thief's questline, and a mage's questline, as well as the typical minor quests from random NPC's. I was disappointed with the Companion's questline in Skyrim after playing Oblivion afterwards. Skyrim's fighter quests were about werewolves AND fighters, whereas I just wanted fighters quest; the werewolves idea could have been an entirely different questline altogether. In Oblivion, I much preferred the Fighter's questline.
I also like the idea of a journal. A book that details everything you've done. They should have an 'encyclopedia' section where you write information about characters, locations, and items for reference. But I also think the journal should document when you level up ("Today I feel like I've learned something new, and my life has new meaning", similar to how the small paragraphs were in Oblivion's level up screen), and document the quests you have done and what happened during them, and events you've witnessed, and it should date each and every documented page. That way, when it's all done and over with, you can read your history and biography of your character!