They'll attack you on site and you won't get to complete a lot of missions.
Also, in my experience, sometimes being vilified also means they'll attack you even if you're dressed the faction's clothing - they just take a little longer to recognise you.
They will also send out hit squads when you are out exploring.