I'm guessing that the Imperials are still trying to maintain a grip on some parts of Skyrim
The Empire isn't only the Imperials. Many of the Nords want to remain a part of the Empire. It's one of the reasons for the civil war.
That brings up a good point though. Would the loyalists have more Imperials or even other races too in their ranks than the rebels? Would the rebels have some sympathetic Imperials on their side, or even spying for them?