That doesn't mean that he liked them. Even if he didn't hate them, most tribes are afraid of, and despise him.
A lot of people serve someone they fear, especially in Fallout. I bet Lanius isn't very popular with the legionaries, yet they obey him. Either way, I hope that tribals in Honest Hearts do serve some kind of purpose, be it serving Graham or fighting him. I don't want any more filler enemies without real cause, like the Raiders in FO3 or the Wildmen in The Pitt. Same goes for the Ghost People in Dead Money to an extent, although they were pretty cool.