Way I see it. The MWBoS fight a bloody war against the Super Mutants and then the Calculator. They force settlements to aid them by giving food, ammo and their best and brightest. Many people do join willingly but the BoS treat them as cannon fodder. The Elders let mutations join because they can be very useful in places humans can't due to radiation.

Vault Zero falls to the BoS and Barnaky becomes one with the Calculator. The BoS are happy to have General Barnaky back. There is a push to make him the head Elder of the MWBoS. The famed warrior (you) who is also a General backs the move to make Barnaky the new head elder, which he does become.

He soon recalls all mutations from the BoS armies and orders them back to their home bases were they are disarmed and tagged, later to be sent to work camps. BoS stops letting mutations join. Laws are passed and any mutation within the BoS' territory is to be rounded up and also sent to camps (work/death camps) which are the start of the great betrayal.

Elders that were most against Barnaky becoming head elder are removed from power. Soon purges happen with in the BoS, anyone that was against Barnaky or seen to aid mutations are also sent to the camps, entire settlement get put to the flame. Soon the Mutant Liberation army forms. BoS caravans and outpost get hit, work camps are liberated, the BoS see a dramatic fall in people willing to join the BoS, and they start having desertions.

Human numbers are replaced more and more by calculator robots and more purges follow fearing an inside job. The BoS stop trying to contact the West in order to get their own house in order.

Biggest blow happens when rouge BoS members set off several bombs within Vault Zero in a failed coup d'état. Many of the calculators systems are damaged beyond repair. Barnaky lives but finds has less control over the robotic forces. He can't control them all at once and the ability to make more robots is severely hampered.

Mutant Liberation Army makes allies with what’s left of the Reaver movement and are supplied with EMP weapons. The remaining slaver/raider groups once again become strong and prey on BoS settlements and outposts.

Decades of bloody Guerrilla warfare against the MLA and the predation of other factions (slavers, raiders) have driven the MWBoS back to their remaining strongholds such as Vault Zero and Cities in between Colorado and Chicago.

Lyons and his BoS are sent out to make contact but don't really know where they are, they know the MWBoS went East, they took a path that takes them away from the MWBoS strongholds at Vault Zero and the others in the Midwest. As Lyons gets closer to DC he hears rumors of people that have similar tech and a symbol are in the area. Lyon spends a long time hunting down those rumors. MWBoS fearing that Lyons is just the tip of a larger army coming up from the south west, do what they can to not be found by them.

Lyons is persistent and soon the MWBoS send out a group of their own elders to make contact. They tell Lyons that MWBoS are just a few hundred men and women close to Chicago. Lyons having made contact goes on to the more important mission of heading to DC.

Like the Barnaky ending states it’s a war the Mutant Liberation Army is distended to lose. MWBoS have seen bitter defeat after bitter defeat for generations and are now a shadow of their glory days but time is on their side. Mutations can't reproduce and the ones that don't fall to the BoS will be taken care of by father time.
Humans, including former BoS that sided with the MLA grow tired of fighting, seeing as how they have crippled the MWBoS and liberated much of their territory. Most just want to settle and farm, the threat is all but gone (so they think). After generations of war, less and less of the younger generations take up the fight.

Maybe the MWBoS will find an ally in Augustus Autumn and his advanced tech can help the MWBoS take back what they lost."

"The hard times" I see for the Midwestern Brotherhood is a decades long war with the Mutant Liberation Army combined with a very paranoid General Barnaky that caused people to turn away from the BoS and ever rebel against the BoS. Along with Slavers, Raiders and Reavers rebuilding their power and causing problems.
The Calculator gets damaged so the Robot Army can't be controlled all at once and new Robots take longer to be built. The Brotherhood is forced back to strongholds across the Midwestern United States.

Midwestern BoS can't produce their own PA so not every member gets one. Only the elite get the best weapons and armour. That is fact.

Enclave would have been worse off than the Midwestern BoS. After the destruction of the Oil Rig and the death of all Enclave leadership but for Eden, much of Enclave moved toward DC, to Raven Rock and Eden. On their way they stop in Chicago, where the Midwestern BoS surround them and make a deal.

The Midwestern Brotherhood, don't know about the Enclave because they left before the events of Fallout 2. They want the Enclave's tech but more so they want the knowledge behind the tech, to save them years of reverse engineering.

“Help us or die where you stand.” The Enclave takes the deal and move on to DC. Only a couple dozen scientists are left behind, which they were going to do any way. The Midwestern BoS give the Enclave supplies and a base in exchange for more advanced tech/weapons. Midwestern BoS make sure they are well guarded.

When Remnants come back from DC after they are once again defeated, they learn more about the Midwestern BoS under Barnaky and their goals. Some see this as a chance to regain some power. A deal is made for Enclave leaders to become elders and the Enclave to remain independent body with in the Brotherhood ranks. Like I.T guys for the Brotherhood.

They help out the scribes and so on developed weapons to fight the enemies of the BoS. The Mutant Liberation army being the most pressing, Enclave leaders now elders start doing what Enclave does best. Become a shadow government with the MWBoS and someday take total control.

Who knows, maybe the Enclave make a deal with Barnaky to help make humanity pure blood again like the Enclave. Maybe that was Augustus' idea all along and why he went against Eden. Maybe he wanted to find a way to make waste landers pure through genetic engineering or breeding. Now he has a willing leader, Barnaky. That is willing to let the Enclave experiment to make humanity pure again, think about it people.