First off, that is stupid. If you are a friend of the legion, you obviously will be a friend of the follower and he will not harm you in any way. A sperm cell could work that out.
Back to the question, I'm not sure about the specifics, but originally ulysses was meant to be a follower, and he had a whole questions with him, unique dialogue, all that good stuff.
Since all the resources are there already, I'm hoping he will come with you somehow after lonesome road.
My Julius is getting lonely, with just a ghoul and a robot to keep him company.
Let's look at it like this. The Legion wants to take over the Mojave, turning every man, woman and child into either a slave, a soldier or dog food.
Helping the citizens of the Mojave (as in, not doing one of the above), is counter-productive to Caesar's goals and therefore likely to get your follower annoyed with you.
And yeah, you're a friend of the Legion. Fine. But you're still a Profligate, as Aurelius of Phoenix, and many others, are more than happy to point out.