Yeah, I mean I'll close the game, then later on, like a few hours later, I would load up Oblivion and it'll work perfectly fine. And I never turn off the computer until I go to sleep, so I don't think it's a case of restarting my computer that fixes it.
I haven't tested it out whether or not if it's just my save or not, but I think these crashes at random places.
Well, the game puts a lot of load on both your CPU and your Graphics card, and the load isn't there when you're not playing it. It might be worth checking with temperature monitoring software, just to see if something is getting really hot. There might be a fan that isn't spinning up, or something like that.
I would definitely try loading a old save, or starting a new game, to eliminate the possibility that the problem is a corrupted save.