I do not think they have the right to commit genocide but that is a very different thing to saying i think they will given the chance, however what evidence have you that they intend to commit genocide?
None whatsoever, yes they kill trespassers but so do people on the surface otherwise we would not have a CW amongst other things and personally i think the risk from the Forsworn is greater and they have every bit as much reason to hate the Nords, particularly Stormcloaks.
Would the Falmer want to conquer Skyrim-don't think so. it would take hundreds if not thousands of years to adapt back to living above ground although they can live in complete darkness their realms are not usually totally dark so they could exist in a vampire like state up above.
Are they intelligent? well i don't know about speech but clearly they can communicate to each other and in all other respects they do everything a Nord does, they farm, they build dwellings, they make things, they smith, they use magica, they keep animals (Chaurus)they kill.
So how does that make them any less intelligent? really i find some of the comments about them on here utterly incomprehensible.
Nords are a race famed for their fighting prowess and intolerance of others so they have picked up quite a few enemies perhaps thirsting for vengeance over the years.
Snow elves, dark elves, elves in general, Thalmor in particular, Forsworn, Argonians, Khajit and more recently the Empire itself.
You reap what you sow.