If all the nords in skyrim decided they wanted the imps ( or anyone ) out, could it be done?

Consider skyrim's harsh weather and environmental conditions.

Might want to go into more detail mate, by "all" Nord's do you mean all the Nord settlements as opposed to just half? They might, that being said half of Skyrim (the Stormcloaks) is having trouble beating a single Imperial Legion (of which there are many Legions, kind of like divisions) so who knows. The Empire's forces are tied up keeping the Thalmor in check, that's why the garrison in Skyrim is so small and why the Stormcloaks haven't been crushed, if this was all happening several centuries earlier when the Empire didn't have another super-power to keep an eye on then they could just have flooded Skyrim with troops.

The reason they're having trouble with one Legion (Isn't it 2 Legions seeing as Tullius brought an extra Legion to quell the Rebellion) is because it's bolstered by many thousands of new Nord recruits/veterans.

If all Nords rebelled then the Imperial Legion would collapse as the Legion historically use Nords as the basis of their armies. Then they wouldn't be able to respond and Skyrim would have a large enough army to tell the Imps to piss off back to Cyrodiil or get massacred.

isn't that the whole point of the civil war questline on teh stormcloak side?

Well I'm not really considering that all the Nord's are linked to some kind of hive mind that means they all rebel. I take it that when the OP said "all Nords" he meant all the Jarl's and their followers, not every single last Nord in all of Tamriel.