Spectacle Island is a funny place. Its happened on more than one occasion that I'll have to turn in a mission to someone (raider trouble or clear ghouls or some other radiant quest) and the person will be out in the middle of the water down on the ocean floor just treading water. Its not even a Spectacle Island settler either, I once had someone from Sanctuary Hills down there.
If you lost a provisioner get a hazmat suit on and go down to the ocean floor, chances are they're paddling around down there. Wait a few hours and they'll usually find their own way up to the coast.
Personally I dont think its worth the risk. Any resources I need on the island I take there manually (strong back perk maxed) and store them all in the workbench. If I need to build at another settlement its no trouble to pick them up from there and just fast travel.
I dont think the build area extends that far into the water to link up those 2 areas. Will need to check that.