hehe, ok, um in fiction, a vampire's hypnosis is fairly rare, in fact any kind of hypnosis is rare. Not alot of vampires (or non-vampires) walking around hypnotizing people, and most probably wouldn't even know the person with the slick hair and black cape in front of them was a vampire to begin with.
But in the ES world charm spells are everywhere. The mages guild has them; the shops have them. So, right away, when someone walks into your shop, you might immediately wonder if this shopper happens to have a charm spell at the ready in order to get a really really good deal on that uber weapon you have.... Know what I mean? But in fiction (in general), not many of people who wander into your shop would you wonder whether they are vampires who happen to have hypnosis ability.
The point is, sure, in fiction people probably wouldn't know they are being hypnotized, because it's fairly rare. But in the ES world, with charms spells widely available, how could they not know? Why would they not do something to put a stop it?
They don't know your charming them (you know, because your CHARMING THEM) and therefore don't realize their charmed and never will. If you really want a system with the possibility of failure then all you can ask for is that Charming someone with a higher Magicka level or skill than you will result in failure and count as assault.
Honestly doesn't matter to me either way.
But that doesn't change the fact that people don't realize their being charmed. Thats the point.