I have Skyrim and Oblivion on Xbox 360 and Morrowind on PC. I like using a controller a bit better, as my hand gets kinda tired from using a keyboard for too long. To me, a controller feels better because it was designed to feel good in your hands(but I may just be spoiled by the XB360 controller). Whereas a keyboard forces your hand to adapt to it's configuration.
Also, If you play on PC, do you plug in a controller or not?
I don't really have a problem with a keyboard except that it gets a little uncomfortable after a few hours of play.