It's a combination of things, but something had to happen in game for me to realize how much I hate them.
So I reluctantly join the RR after walking the first time Dez gave me the ultimatum of "free synths, no questions asked, or leave". But dat Ballistic Weave...
Didn't help that Deacon was all like, "Please brah, join us. You're a badass, we need you!" So I acquiesce and chaperon him on his little trip to the Switchboard. Mind this is after meeting my disappointment of a son, Shaun/Father/Director, who's responsible for murder, kidnapping, fraud, and unethical scientific experimentation on an astronomical scale.
When I get back Dez tells me to pick a code name, I'm like whatever, and she calls me the Wanderer. Alright, cool.
Then she says "we know" and right then and there asks me to betray the same Brotherhood that's had my back for the whole game for the sake of sophisticated vibrators/flesh lights. I'm thinking to myself "Yeah, she's crazy. I'll ignore her and see what else I can do".
Carrington then sends me to a dead drop in the middle of a Super Mutant Master stomping ground. As I'm getting my ass handed to me in the name of Freedom for Fisto the sixbot a vertibird flies overhead in all it's minigun glory. "I've got your back Brother" is what I hear as two Paladins drop from the vertibird and turns the colony of Super Mutants to ash.
That right there was all I needed to see. Why should I carry a pack of whiners on my back for the sake of their well-intentioned misguided emotional stupidity when I can be helping the Brotherhood protect the Commonwealth from real threats.