Have you ever taken a slope with a horse? The things can gun a near sheer cliff rather fluidly. Their endurance and intelligence may be questionable, but you have to respect that they are so ignorant as to defy gravity.
Agreed! This is the only reason I've ever bought or stolen a horse. Climbing mountains by simply running straight up them has gotten me to some awesome locations, some of which weren't intentionally accessible. Otherwise, they're useless. I can pop stamina potions and keep up with a galloping horse indefinitely, and I don't need a furry war pony attacking that angry Ancient dragon that has decided to swoop out of the mountains to feast on us mere mortals. I can kill it by myself, thank you very much. Also, it's much more fun to run around on foot and explore, rather than sit atop a steed whilst zipping past all the cool random stuff on the ground. And, as others have stated, dismounting during combat is painful. It's like, yeah, sure, you're dragonborn, but have a sense of urgency, will you?