Am I to understand that if the Ferryman at Cottonwood Cove is for some reason killed before you partake in "The House Always Wins II" there is no way to get to the Fort?

I think this applies, from the vault:

If you take out the entire Cottonwood Cove Legion contingent in the quest Eye for an Eye, even after getting an invite to The Fort, this ferryman will be nowhere to be found, you still can get on the craft and travel on your own

swim there?

boy do i feel stupid....thank you