For one thing TES is like an incredibly fun Sandbox to me, in which I can RP, goof-off and just have a good time, immersing myself completely. In doing this, an easy game is far more fun. However, I also like challenging games. I like games that make me constantly save for fear of dying. I like games that I always have the fear of being overrun. I like games that have boss-fights that take me 5 tries to win.
So I'm happy either way difficulty-wsie.