As a workaround, once you find your followers, this is what I do:
1. One of my settlements does not have a recruiting beacon (Note: Nordhagen Beach).
2. Another settlement is my "follower" settlement where I retire them once I have their perk. (Note: Red Rocket)
3. Any time I'm ready to 'retire' a follower I send some random settler from Red Rocket to Nordhagen Beach which isn't at capacity since there's no beacon.
4. Tell new follower to follow me.
5. Send old follower to Red Rocket.
This way they're all at the same place and I can go pick up Dogmeat and 'solo' the wastelands once I've got all the perks.
As a suggestion if you've got a big settlement like Sanctuary you're having trouble searching, build a bell. You can summon all your settlers with it and see who all is at that one.