I haven't been playing them since 1998, but I did play Morrowind when it first came out, and I've recently started Daggerfall. Morrowind's my favourite because it was an explosion of awesome to the power of >9000, so it'd be easier to say what was wrong with the other 2.
Oblivion was great, but it lacked the sense of lore. The combat went from simplistic to halfway between simplistic and action, which really didn't fit - it just ended up being clunky. This generally represents the feel of the game - very grand, but without much good substance.
Daggerfall I'd say isn't the best is because of how many restrictions it places on the player. For example, every quest, even the Main Quest, sets a time limit on the player. This means that you'll have to turn down a bunch of quests purely because you can't do one of the awesome mega-dungeons in the time you have left. This 'restriction' quality, as in Oblivion, ricochets throughout the whole game - for example, it's incredibly difficult to play a fighter (or any non-mage class for that matter), most dungeons will have extremely cheap enemies (like the paralyzing spiders) and you have to get a certain weapon before you can even hit them.
And Arena?
...yeah, we kind of ignore that one
Morrowind is the best, it was immersive, difficult, had a good story-line, and it had good customization. It will definitely be tough for Skyrim to dethrone it, that's for sure.
You say it was difficult, but honestly I think its difficulty was somewhere in the middle - not too hard, not too easy. I wasn't dying constantly like what sometimes happened in Oblivion at later levels, and in Daggerfall at all, but I was still having a fun, challenging time. Because, to put it bluntly, there's nothing that's more jarring during a dungeon crawl than death.