I've had this bug for quite some time and for whatever reason it still happens across all of my new computers i've gotten throughout the years. After completing the Oblivion gate at Kvatch and getting Martin to come along with me I encountered an Imperial Guard along the road away from Kvatch (I forget whether he was running or riding a horse) who immediately comes up and arrests me for no reason. Any way to fix this?

Is your bounty 40 gold? In some games, attacking a Dremora before he attacks you may be counted as assault. This is a very annoying glitch. Don't know how to fix it, but if you're on PC, you could just reset your crimegold.

You also might have accidentally hit one of the guards when you were assaulting the Daedra outside of the Chapel.

Probably got arrested for killing a Dremora then. I'll be sure to reset my crime gold after closing the gate.