A better GPU would certainly help, you certainly don't need an expensive one since oblivion is not young anymore.
What GPU are you currently using?
One thing is sure, RAM is not the problem here.
Hmm...You should have no problems with that GPU. I Personally Had that gpu for almost 5 years and with a heavily modified oblivion i was never able to get low fps or any irregularities.
Could you post your full specs?
Does your pc have adequate cooling and free of dust?