Well, I do use VATS as much as I can with pistols, but for quite a few fights I've blown through my AP and would have to freehand it for cleanup. I used the MM for:
Spoiler The assassin attacks at the White Glove casino, as well as for the ambush Benny leaves you at The Tops.
In both cases, I had to resort to free aiming to finish the job. In the first case, I even used all six shots for a single attacker and had to reload. It was very strange.
Yeah, sometimes VATS is glitchy, and can waste your AP without much gain. That's why I always hit quicksave before upcoming fights, at least when I know they're coming. Bless that quicksave button! I do so hate games that only use savepoints, and don't let you choose when and where to save. If something buggy and just plain wrong happens in a fight, that's the fault of the game engine and not me, I don't hesitate to call a do-over.
There are a lot of nuances to combat in NV, and one of them is the 'who gets the drop on who' factor. In fights like the ones you describe, where they get the initial drop on you, and you have no sneak advantage to start with, and you don't get to open it up with one or more sneak-crit kills, then yeah, those types of fights can be rough with a 6-shooter. That's why you keep something else handy to quickly swap to for those situations, like a good shotgun or SMG or assault carbine, etc. No one kind of weapon is always the right tool for the job.
Don't forget, when you run out of ammo with a weapon that has a slow-reload time, it's always faster to quick-switch to another weapon and keep on firing.