Modded for extra speed and unlimited sprinting, they're fun to ride and make a good alternative to fast travel. But my computer can't really handle having to load and unload cells and textures at higher speed, so I get a framerate drop and frequent ctds. My current character just uses carriages and jogs from the nearest city. If the performance and speed issues were fixed, I'd use them all the time. Some other things that would help would be:
1st person horseback riding
adjustable AI aggression settings (Feel free to charge in there when I'm a low level archer that needs a tank, but just stay put once I can take care of myself. And in either case, go back to where I dismounted once combat is over.)
horseback combat
horses rideable by companions
horseback ingredient picking
horseback dialog
in roughly that order.