Well, I have a pretty high end system, but in odd spots, my framerate drops down extremely low. Sometimes into the 20s

Some examples.
http://i.imgur.com/4gUc8.jpg
http://i.imgur.com/uyJ29.jpg

Drops from 60 to in the 30s/40s simply by looking in a certain direction when there's nothing going on. The CPU and GPU usage aren't maxed either, so I don't see why it's doing it.

I wouldn't mind so much if it didn't cause the game to stutter like mad and feel sluggish.

Specs:

AMD Phenom II x4 955
4GB DDR3 ram
GTX 480 (277.33)
Win 7 64-bit

Settings are 1920x1080, 2x AA, max AF and everything else on max, which is the recommended. Same happens with AA off.

What do?