To approach something like knocking people out, you have to look at reality first and then try to emulate it in the game. Sleeping magic can factor in later. Also, I think the justice system needs revamping.

If I'm walking along and someone is running at me with a sword, I'm going to arm myself and fight back. The chances of knocking them out with a mace or something aren't great because in real life the mace would crush the skull (yes, I'm mostly assuming). However, there is a chance during the fight that my weapon hits the right part of their head and they fall down unconscious, but the chances of me doing this intentionally are quite low unless I'm very skilled, or using a big stick. It's far more likely that offered the chance to yield, in their battered state, they'd accept. Once yielded, I should be able to knock them out by some means if I want to carry on with my business or simply for fun, or at that point induce some kind of sleep spell/alternative to chlorophyll or gym socks.

Most of the weight of these instant knock outs should come from your sneak ability and hand to hand. I don't know how you'd calculate it for weapons, like smoking someone with the flat of a sword, but the whole butting someone is kind of lame for a knock out since it doesn't work like that and it's cheesy to see. If it would be too easy to dispatch certain people or something, make them harder to sneak up on. If you want to knock people out, you should have to use hand to hand, or a blunt non-mace weapon (like a baton) that you have to switch to because hitting someone with the side of an axe will probably kill them. As a sneaky spellcaster, you can either use your staff, or whisper sweat nothings into their ear. It shouldn't be a ranged "attack" to put people to sleep, or if it is, it should have very low probability of working on higher levels/wisdom/intelligence or something, and should be most effective if your presence is secret, I think the chance of success should be dramatically reduced for semi-alerted enemies and near impossible when someone's attacking you unless it's a very low level thing vs a very high level mage.

Someone said you can't have the same metal gear type sneak stuff in a game like Skyrim but I don't understand why this wouldn't be great for a world like TES?

The justice system needs an overhaul, because if I'm alone in the woods and there's one other guy near me, I should be able to kill him or knock him out (without his knowing or seeing me) and the lack of evidence should absolve me from crime. Why not? I wouldn't play the game much differently if it were that way, it would simply feel more authentic. If I attack some guy in the middle of the woods and he starts screaming and running away but my arrow catches him betwixt the shoulder blades, then tough fuzz buddy, you're a dead man and I'm Scott free. That's how it would work in that world. Now, if he manages to escape and gets to a town or guard, that's a different story, then I'm wanted for assault and guards should be looking for me. Conversely, if it's a guard that I'm attacking, nobody should know what's going down, unless there's someone else there to report it. If it's too much to ask guards to move in small groups of three or something, then at least make it so one other person has to witness it, or know of it, before it becomes a crime. Then if you simply go around putting people to sleep while they're aware of who you are and you take their stuff, someone is going to know about it and report you. The knocking out should work like the Hitman games. If they know it's you, then you're blown, if they don't know, they don't know, but they'll tell someone and everyone will be on alert for some guy going around knocking people out. And an hour is way too long. If I go up and strangle some passerby, they'll be out for maybe ten minutes or something - if that. It should be a way to deal with someone quickly without killing them, to get by, just like metal gear or whatever. Of course potions should come into play, and you can give them a little something on the ground or in lieu of choking and it'll put them out for an hour.