Sneak attacking with a melee weapon does 4x damage. Sneak attacking with a ranged weapon does 1.5x damage. If you are undetected by your target at the point at which you hit them, you should always get a critical hit - the only 'chance' is the normal chance to actually hit your target. It is possible to miss and still get the 'critical hit' message, but this is due to a bug in the game engine.
Although I don't always get critical hits when I'm sneaking, seems fairly unreliable. It does work with bows but is much less powerful, for good reason of course.
Basically, even if you've got the sneak icon, that second when you raise your weapon or whatever gives them a seemingly higher chance to notice you because you're making more noise or something IDK, and you have to be unnoticed on impact.