Nor would you gain any experience from fighting enemies. Nor would you gain any possible alchemy ingredients. Nor would you find any loot. Nor could you discover any locations. Nor could you find anything to gain besides the time to speed up travel (real time).
You're perfectly right. It would be wonderful if based on your skills, the game would randomly calculate some minor increases for things you are likely to use (Bowskill, Longsword Skill, Heavy armor skill) and then also give a chance to add to your inventory items you might pick up. This could be established in such as way as to create an interesting narrative for each of your travels, such as "You arrive in Winterhold. On your way you used 5 arrows, added slight wear to your bow, and gained venison." Or "Your arrive in Winterhold. On your way your Heavy armor skill increased, your Steel longsword received minor wear, and you gained 10 gold and a silver dagger." You can see how these easily represent their own adventures, such as hunting a deer or encountering a bandit.
Of course, it is all rather complicated, and much more easily represented by loss of gold.
I just don't get why you need a "fast travel" option when previous games already gave you transport along with teleportation spells and scrolls. The means to travel quickly (in the game and out of it) are already in place. I just feel it is another simplification. Is it tedious? Yes, but so is inventory management, and repairing weapons, and having to restore your health. They are the nasty little parts of the game that make the fun parts seem comparably more fun.
Also, Oblivion's problem was that because fast travel was so inherent to their system, they thought nothing of having the Fighter's Guild in Cheydinhal sending you on a mission in Anvil. Ridiculous. I NEVER had a problem traveling in Morrowind, either using the systems or walking. In fact, I LOVED the "journey" aspect of taking a silt strider to a port town, taking a boat to another city up the coast, wandering into the wilderness, getting lost, then hoofing it to some minor settlement to unload gear, repair and refresh.
But each time I tried to avoid using fast travel in Oblivion, I got INCREDIBLY frustrated, partially because there was no fast travel outside of the "menu teleport" so I was walking/riding everywhere, and because there was no mounted combat, I found myself dismounting every 2 minutes, OR leading a circus train of lions, tigers, and bears, OH MY. Nevermind the above problem of having quest which send you across the map.