Just say that Caesar's Legion didn't push their army west, but instead went east. They cut through the Midwest Brotherhood of Steel like butter (or maybe they went the same route as Lyon's BoS, finding nothing but tribals and petty raider gangs), and eventually find themselves standing http://images.wikia.com/fallout/images/f/f0/WelcometothePitt.JPG.
Let's assume that the Legion doesn't cross the bridge into The Pitt guns blazing, but instead Caesar sends his Frumentarii into town, where they meet with Ishmael Ashur.
What happens next?