What animals from the TES lore do you like the most, that would be suitable for riding? Besides horses, of course. My favorite so far is the Guar. Man, I'd love to ride one of those in Skyrim. They look so strong and probably intelligent since they can be domesticated.
Game wise, it would be a nice touch to see npcs (traders, adventurers, military units) from other provinces riding wild creatures that are not common in the current province where the game is set. The player could even buy or steal one, and be the owner of a very rare sight that local people will admire or fear. Each mount would have different attributes, strengths and weaknesses depending on how well it can adapt to the local climate and food.