It works fine for creatures, magic resist will decrease the effect. However, high magnitudes will cause some strange abnormalities since Flee AI will rollover to 0 past 255. Can't think of a way to get it to work on vampires though, since turn undead doesn't effect them in any way (could argue that they're still protected by Arkay's Law). NPCs can't cast the spell either it seems.
Anyone interesting in this work? It makes turn undead fairly useful for tomb crawlers, and the Ritual can now slay the undead with impunity. It rebalances the spell costs and available spells, and it'll add a script to all undead creatures, and require a patch to add the script to new undead creatures.