As a RR guy, I had this thought myself. I would have to ask more lore knowledgeable people, but does the Brotherhood in the Commonwealth represent THE Brotherhood very well? By that I mean, was Maxson the de facto leader, such that his death is going to prompt a vigorous military response? Or is it possible that his death will leave a power vacuum that they will have to struggle to fill? Also, did the Prydwen represent a sizable commitment of Brotherhood force or was this the tip of the spear? Is there a Prydwen 2 or any other aerial based transport (besides Vertibirds) that the Brotherhood has?
Knowing how the Brotherhood rolls, I have a feeling that a diplomatic response would be slim, but if there isn't another Prydwen it would have to be a ground based invasion. The Railroad has somewhat exposed themselves, so if the Brotherhood comes for blood, they will go back underground. Depending on if they can recover the location of the church from the archive on the Prydwen (or if that broadcast that information) they might have to relocate their base as well. From there on out, it becomes a guerrilla warfare thing, and it just remains to be seen whose will and resources will last. I feel like the Brotherhood would have to tap the local population to sustain their war effort and that would burn through any good will they had left. Forced occupation doesn't endear you to a populace.
Furthermore, the main reason the BoS came to the Commonwealth was to destroy the Institute. Since that goal IS accomplished, I have a feeling that whoever takes over the mantle would not be so eager to jump back into the Commonwealth in force, just to go after a rag tag group. They might try, but as I said above, I feel like the miseries they would incur in such a campaign would be such that they wouldn't bother to see it through. I also firmly believe that one of the natural outcomes of the RR is going to be them reformatting themselves. I could see them becoming some secretive human/synth organization that works with the Minutemen or becomes infused into the Minutemen. The RR would then be the brains and intelligence portion of the Commonwealth and the Minutemen would be the force. If that holds true, then I think the BoS would have an even harder time faced with such an organization.
Again, most of my thoughts so far rest on how big the BoS is and how they are going to respond to this kind of attack. My final thought however, is how the attack might be construed. If, and this is a big if, but if they send an small recon team to the Commonwealth for information or if they even agree to meet with the leaders of the Railroad, then we might be able to make a peace deal centered around the fact that the BoS struck the Railroad first and with deadly force. I still do not understand why the BoS did this. Yes, the RR is helping synths escape and they probably wouldn't be best friends in any case, but whatever happened to the enemy of my enemy? Most importantly, it seems like a horrific military blunder to attack the small fish when you have much bigger fish to fry, ESPECIALLY because once the Institute was out of the picture, the Railroad would no longer have a reason to exist. How could Maxson not see this? If he is so rabid about their destruction that he must necessarily track down and destroy each and every single synth, then I guess it makes more sense, but that is just incredibly stupid. Especially since he has one serving under his nose for the entirety of the game as his right hand man lol. A good leader looks at how to achieve his goal with the least expenditure of blood and resources. Maxson's youth and naivety show readily throughout his entire campaign.
edit:
I should give Maxson a little credit in that he was also in the Commonwealth to eradicate Super Mutants, Ferals, and other nasties, and the Brotherhood is constantly engaged with these groups when they are in the Commonwealth. I agree fully with Danse when he talks about wiping them out in Fort Strong. However, that goal is very much tangential to the primary goal of the BoS in the Commonwealth and still doesn't excuse some of the tactical decisions that Maxson makes.
Also, I know for my part personally, I killed Maxson before I left the Prydwen. I'm surprised you didn't, as the pathway out of the ship leads you right past him and he has some powerful guys at his back. I wouldn't have made it off the ship alive without killing him first. He would've mowed me down.
But I digress. Those are my thoughts on how it would resolve itself. Either way, a full scale invasion would be nasty business. It would be tense as well. I don't see a clear path for the Brotherhood to simply come in and clean house though. I suppose if they had the location of the HQ and they just nuked the place, that would be a decisive sort of revenge pathway, but that would be the only decision I see that is feasible. It sends a message, it repays the loss of the Prydwen, and it requires very little resources from the Brotherhood itself.