One of the first settlements I unlocked was Abernathy Farm (via the quest returning the favor).

Several dozens of hours later, sure enough, returning the favor is now active again in my quest log, and I no longer have access to the workshop there. Is this a bug? Or can you actually lose settlements if you dont pay enough attention to them?

Settlements that have pre-existing population that you have to "ally" with, can decide to leave you if you don't pay attention to them. Abernathy farm is one of those.