I can see how you arrived at that conclusion, OP, but please consider this:
A charm spell is an Out Of Combat spell - for getting more information out of a npc, for getting a better price from a shopkeeper. Making it a outright crime would kind of render it useless, or at least impractical.
I can understand if you try to charm-spell an expert mage who could understand what it was and the intent behind it, if they become hostile, but not unless.
As for buffing yourself in public being illegal - ok, consider this. You're walking through town, you've accepted a quest to clear out a house that's infested with mud crabs. You buff yourself for extra health - and get promptly assailed by "STOP! YOU'VE VIOLATED THE LAW! YOUR STOLEN GOODS ARE NOW FORFEIT!"
Impractical, maybe. The same way that murdering your way into the Redoran vaults in Morrowind was impractical.
Using a Charm spell on someone all alone, or with no one looking, or with people to busy to notice- that would probably be overlooked as just a gesture in conversation. But if you shoot a Charm-on-target spell at someone in front of a guard captain, naturally he's going to ask "the hell are you doing?" It's about being subtle.
Also, placing buffs on yourself on a public place wouldn't be outright illegal. It would simply place you under suspicion. The same if you draw your weapon in a tavern, tail someone in sneak mode, and so on.
If you were to cast a buff in a training area or a combat situation, it would just be common sense and no one would pay any mind.