I tried to get a follower of mine to follow me recently and could not do it, then tried to hire a mercenary and she told me that I already had a follower. But I don't know who or were they would be. Do I really need to systematically search? Is there a way to get him to come to me?

what system are you on?

PS3

Well, good luck.

Reload an old save from where you last had your follower with you.

Have you tried fast traveling or sleeping for several hours? Sometimes my "lost" followers will appear after a long nights sleep. Sometimes it just takes a while for them to catch up with you.

PS3? you might be out of luck. Either your follower got stuck somewhere or died. Fast travel "should" have mapped them to your location. Unless they are dead. Wait several days. If your follower is stuck somewhere either they will catch up or hopefully auto dismiss. If the follower is dead, the same thing should hopefully happen and they auto dismiss.

Otherwise if you have any quests that force a follower with you, it should unfollow your current one. example being