And to throw a wrench into the works, there is also biological attraction. I've heard the reason some peoples B.O. (Or they just feel "off".) even straight out of a bath may be off putting to someone is you share much of the same DNA and it's natures way of saying "find someone else". Not that I have proof to back it up, it makes sense.
The "feeling off" thing I get a lot. Not just with women, but men as well. No matter how they behave, I've met quite a number of people in my short time that I've immediately disliked. What's more, those who I've tried to become friends with have all done things to really regret that decision.
And there are girls that, even though I know they aren't utterly stunning, I can fall head-over-heels for just meeting them. There's definitely of the subconscious than the conscious with attraction.