Horses are categorically useless in this game, to be honest. They would be useful if they could... hold objects in a saddle bag, or... run faster than the player, or... would stay put if you wanted them to, or... you could fight or cast while mounted, or... the longer you owned a horse the smarter/faster/whatever it got.
Yes, I know they eliminate encumbrance penalties, but honestly, are you going to crawl for 20-30 minutes at a snail's pace out of the bottom of a dungeon to jump on your horse just to enjoy the normal movement speed? I also know that some horses are better fighters, but still, it's nominal. My experience has been that horses are basically vanity items.