I believe the pardon is the least of anyone's concerns. Given you aren't even on any list in the Empire. The pardon is completely and utterly pointless and worthless. In other words, its a bad argument point seeing as the only people who remember you are the SCs (who won't sell you out to the Empire) and the Imperials (Tullius barely remembers you and Hadvar is the only one who probably cares to remember you).

That just makes the fact that you have to earn a pardon even more ridiculous.