By the time I enter the Castle of Kvatch, it turns black and I get an window saying that Oblivion is not responding. (Yes I have a Windows 7) Then it closes Oblivion and when I get on Oblivion once more, the autosave of the place I've entered that closed me out said "The Save Data Is Corrupted".

That happens a lot with Autosaves. Never rely on them. Your only solution, I suspect, is to reload an earlier save.