As has been said, it's not hardware limitations you're up against. It's a crappy engine. One that simply can't handle being turned up to insane levels like you're wanting to do. I think you should probably reset your ini file back to vanilla first of all, and then tweak downward, NOT upward, from there.
Well, I installed QTP3 Redimized and gained about 3 fps. Reseting my config file gave me 4-5fps extra. After increasing view distances, fades, shadowing in the IG menu to max I only had a 1-2fps gain over previous .ini and 100 more free vram.
Probably the uGridsDistantCount, which is far more painfully deadly than most other things you could do to the game.
My uGridDistanceCount was at the default 25, changing it to 16 was the first step I tried in this thread, Tree count is also at 16. Tomorrow I"ll fool around tweaking different things while monitoring memory/cpu/gpu to get a more, thorough, idea of how things are impacting performance.
I was afraid it was just the engine that was limiting performance, a bit obvious considering there's a gig of free ram and processing isn't being fully used, but I was trying not to entertain this in the hope that there might have been a straightforward solution to this problem. The most I suppose I can hope for atm is to find a way to squeeze enough performance out of it to get a stable 24-30fps so that motion is at least fluid.