Very infrequently.
In Oblivion, I usually used fast travel if I forgot something. Like if I was journeying from Anvil to Leyawiin, and somewhere around Skingrad I remembered that I forgot something, I would fast travel back to Anvil, grab what I forgot, then fast travel back to Skingrad and continue on. I enjoy the environment, the music, seeing the cities off in the distance. I guess in short, I enjoy traveling everywhere
I've still used it when I didn't have to, but I am typically in no hurry playing TES.
In contrast, I used it all the time in New Vegas because I didn't care much for the game, so I was just trying to blow through it and finish it. But in a game that I really like, like TES, I want the game to last as long as possible.