I agree. I think companions should work like Martin in Oblivion. If you deviated from the path of the main quest, he would still follow you on his horse and dismount when you entered a city or building. It's silly to have them running behind you all the time and getting lost. It would really add a lot to the game if they could ride with you.
Is there a mod for this on the PC version? I'm considering switching from the 360.