So I never built a radio beacon there and just powered up the massive beacon that's part of the quest. Problem is that all my other settlements are attracting settlers except Castle. I got a few extra minutemen upon returning there the first time but since then nothing. So I am wondering if I leave it as is if I will attract more minutement, settlers, or both or if I won't attract anyone and need to build a seperate radio beacon?

Yeah, you need to build a Recruitment Radio Beacon. The one that's there is only for running the radio station.

I would assume you need a beacon, the quest was all about powering up a radio station to help you get minuteman quests, it has nothing to do about attracting settlers.

That is admittedly unintuitive. But yeah. You need a separate recruitment beacon. Or to manually import settlers from other settlements.

You will probably be glad of it however when you either get tired of the mission spam and want ti turn off Radio freedom while still recruiting or have more settlers than you have jobs fro and want to stop recruiting without necessarily loosing radio freedom. Having separate towers lets you do that.