I use Fast Travel quite a lot, but I also walk quite a lot, it's kinda 50/50 with me.
But I always fast travel to that place with the Greybeards and places that are hard to reach.
Yes, that place is hard to reach and should have had other means than just walking/riding to be able to access after the first pilgrimage.
I'm lazy and whenever there is an easy to use fast travel system, I gradually get drawn to it and loose a lot of my connection with the game, as I did in Oblivion.
I just wished that they would implement something like what I have suggested
http://www.gamesas.com/index.php?/topic/1207074-how-to-have-a-cake-and-eat-it-too, and I would probably mod it in at the first opportunity.
And the High Hrothgar? problem should be solved by something like learning a spell/power/shout from the grey beards that would transport you to an altar there, whenever you cast it.