After trying to narrow down which mod was causing it, it was still happening with absolutely no mods active at all. My load order consisted of Oblivion.esm and nothing else, and I was still moving too fast.
So I'm going to try and reinstall Oblivion, but I want to keep all my mods. What's the best way to do this? Do I need to move them to a separate folder?