I would agree that many are ignorant, not because they dislike the Legion but because they refuse to even listen to what they have to say.
I wouldn't say that as it would no doubt start a flame war.
I've listened to what the Legion has to say. They believe that technology led the world into decay, and that decipline and devotion are superior to technology. Caesar believes that the NCR is weak becuase they offer personal freedom over security and military might. He believes that they have become corupt and that his Legion is the answering force that will remove said corruption from the wastes.
While most of this is fairly accurate, Technology is a nice crutch in place of dicipline, and that the NCR has become corrupt (Oliver and Kimbel are evidence of that) and offers more personal freedom than security and strength. This does not make me want to join the Legion, nor does it make me think they are in the least bit acceptable.
The Legion belives that only the strong deserve rights. If you're not a capable fighter, and male, you are either killed or turned into a slave. Those who do not follow their strict guidlines are executed. they belive that they can take what they want simply because they are stronger. Yes, the wasteland is a dog eat dog world, but the Legion is adding to the problem rather than being part of the solution.
Also, as far as I know, they do not accept ghouls or super mutants, which is not okay. All in all, that makes the Legion misogynists, facists and purists. They've hit the trifecta of evil and for that basic reason they need to be destroyed. The Legion represents the eventual stagnation of the human race, and a backward decline into barbarism.