Have you noticed that your shots will often miss the target in VATS despite 95 % chance? This usually happens when enemies pick up weapons from the ground or move around the corners or try to take some cover, when NPCs are moving in FNV, they are kind of flying thru the air, they don't even seem to be touching ground and cover great distances with a single jump of 50 metres, VATS simply stops working in such situations, bullets fly thru enemies, without hurting them. There were some problems with projectiles in F3, so I tested different projectiles, changed projectile animation, speed etc. and I noticed that beam projectiles like Tesla cannon projectile or the pulse rifle projectile are less prone to this bug, while bullets constantly miss anything. So I went ahead and changed the Cowboy Repeater projectile, it's now a beam, like the Tesla Cannon, using same animation file and moving at lower speed, because the original projectile never hit anything when enemies were moving, like you have an enemy 5 metres away from you, you shoot him in the face, nothing happens, bullet flies thru him, hardly ever happens with the tesla projectile:

http://i467.photobucket.com/albums/rr40/WRFan/ScreenShot55.jpg

Vats is a pain in the butt.
Your issues you described are only the tip of the iceberg for me...
I have pretty much given up on Vats, because it svcks.

I now manually aim, preferring high velocity projectiles.
Melée/unarmed works best for close range- so npcs can't dogge your shots when they power attack.