Well, for the uninitiated to the Elder Scrolls Universe, there's probably a bleed effect happening when they see the Beast Races and assume "Furries". Also, from a in-game perspective, the Beast races have long been persecuted and enslaved.
For me personally, the Khajiit and especially the Argonians are so alien to me and my thought processes I can't imagine playing one for extended periods of time. I always roll Argonians and wander away from them in hours.