Fast travel, in itself, isn't an inherently bad thing. The problem it has in Oblivion is that it was poorly implemented, for a game world that didn't really need it, and then quests were shaped to rely on it. It doesn't much work to imagine that it's walking, or boat-riding, nor does it help for the sake of immersion to go out of the way to pretend these things (making a point of walking to a coast town
just to fast travel to another coast town would likely break immersion more than the fast travel itself). At the very least I dislike it because it's an imbalancing factor; you are guaranteed to reach your destination, with zero danger or variation, and you heal as though resting along the way. That doesn't exactly invoke any feeling of exploration or potentially endangering yourself forging out into places unknown.
I think "the world" is an important part of a sandbox roleplaying game. If you want to roleplay a character, after all, you need a universe to move him around in. Fast travel is fine when warranted, such as in Daggerfall, when nobody wants to spend the rest of the day holding the forward button as they run across featureless wilderness for hours and hours and hours. When it has a cost to resources and safety, however, you can't just casually zing all over the map wherever and whenever you want. Since Oblivion's version has no cost, and quests frequently send you across the world, you're encouraged to fast travel all over the place. As a result, you don't see much of "the world." It creates a sense of detachment that you can never really shake, since the free, flawless travel is always floating over your shoulder. There is little sense of danger when an unremovable lifeline is tied around your waist.
I'd say there's certainly room for it in TESV, especially if the map is bigger, but it needs improvements to gameplay and balance features so that it
fits, instead of hanging over you like an awkward cloud. It's supposed to simulate walking, and walking is supposed to be the
least convenient form of transportation. That's why they bothered to invent all those other kinds. Really, it comes down to something similar to the complexity vs streamlining thing; fast travel is a time-saver. Saving time is good, like streamlining, you just have to be careful that it doesn't strip away more than time.
It would be cool, but it might take up too much time and I don't want it in TES V if it hinders the game in some other way.
I don't think it would be much hindrance. Obviously we're going to have moving objects and collision physics in the next game, so being able to sit in/on something as it travels is hardly a stretch. Look at the imperial patrols in Oblivion; we've already got "vehicles" going up and down the roads, just stick a carriage on them and let the player sit inside. When you're paying for the ride it would be easy to give a basic dialogue option, like "I'm in a hurry/take the scenic route" or something, so the player can choose between appearing at the destination or watching the ride.
It comes to mind that it could take another simple step toward realism there. Talking to other passengers would be an ideal time for gathering rumors and recent news, since that's exactly the kind of place where smalltalk takes place. Certainly makes more sense than getting a casual, informative response when you approach a stranger in the street and ask them for "rumors."