Ultimately, to me, it seems like the Stormcloaks are way, way, in the wrong. Wall of Text incoming.
First off - the empire isn't as 'repressive' and 'corrupt' as people paint it to be. The empire HATES that it was forced to stop worshipping Talos, beyond a few twits. On both the stormcloak and imperial side you see indications that many in the empire secretly worship him. The only reason they banned public worship is because they had the crap beaten out of them, and it was better to accept a temporary banning of public worship than have a collapse of the only major human nation in the world. They're not 'invaders' in skyrim, either - they are the motherland of the empire, and the person they worship became a god in part because of his role as the first Emperor who united Tamriel. For some reason, the Stormcloaks have decided to blame the Empire for losing a war, even though the armies of Skyrim were right along side the armies of the rest of the Empire.
As far as corruption goes - compare Imperial cities and Stormcloak cities. Imperials have Solitude, which is overall pretty content and happy, without any obvious injustices, barring the fact that they executed a soldier who disobeyed orders (ultimately, being a soldier in an army means that it's a crime to decide based on your personal beliefs to aid an enemy - it's one thing if they were beheading random civilians, but he was a soldier). There's Markarth, where there's a huge, massive criminal conspiracy involving mass murder and slavery.. run by Stormcloaks. There's Falkreath, which is pretty much a nothing - nothing particularly good or bad about that place, but that's a good thing, no corruption or horrible abuse or anything. There's (technically) whiterun, which is one of the more pleasant places in Skyrim. All the cities are fairly upstanding and just, without anything horrid going on, except for Markarth, where the biggest criminals are Stormcloaks.
Now, look at Stormcloak cities. Whitehelm, where Argonians and presumably Khajit aren't allowed in the walls, Dunmer live in slums and are regularly harassed, Nord merchants are protected by guards, while non-Nords are left to be killed by bandits. Even a few Nords go to Ulfric to try and have him help stop these injustices - he ignores them entirely and refuses to meet with them. There's Riften, which.. well.. nothing really needs to be said about - the entire city is corrupt to the bone. There's Dawnstar, where an abusive, corrupt Jarl is hated by the populace and actively discriminates against Imperials. There's Winterhold, where nothing particularly bad happens, but the Jarl is a [censored].
Which one seems more corrupt to you?
Then there's the fact that the Empire was getting ready and building up an army for Round Two with the Thalmor, when the Thalmor deliberately manipulated events in Skyrim to cause Ulfric to start a rebellion, further weakening the empire and preventing them from striking back against the empire. Why would you do this? Can't you hold your massive ego in check for a little and stop [censored]ing about religious freedom and revolutions until AFTER the genocidal elves that are the reason you don't even have religious freedom in the first place are gone? He wants to separate from the Empire to fight the Thalmor, but if Skyrim, Hammerfell, Cyrodiil, and High Rock couldn't win against the Thalmor together, why would Skyrim do much better alone? It seems to me like the result of Nordic arrogance - they're so much convinced they're better than everyone else that they somehow think they can take on the whole world by themselves despite evidence to the contrary. The fact that they fear and loathe magic means they'd be at an even further military disadvantage, presumably high elven magical supremacy is what's giving them such a large edge in the war (I can't imagine High Elves winning due to being the best fighters).
Then, there's the fact that the Stormcloaks don't represent the entire populace of Skyrim. The Empire is generally inclusive, while the Stormcloaks are exclusive. The empire will accept almost anyone as a citizen, and generally seems a pretty tolerant place. Stormcloaks say it again and again : "Skyrim for the Nords!" - they've got a very 'us vs them' mentality, (they even say it themselves "if you're not with us, you're against us" and the only time they'll ever accept an outsider is if an outsider is either rich and powerful, or willing to fight for them (probably as cannon fodder on the front lines). The Dunmer are blamed for not fighting against the Empire - I notice that Nords (and humans in general, really) aren't blamed for staying civilians instead of heading off to fight on the front lines. Ultimately, with that outlook, people who supported the empire, or just stayed out of it and tried to care for their families, will be blamed and probably forced to leave, or live as lower-class citizens.. if racial violence doesn't start. There are a lot of minorities who have lived in Skyrim, some their entire lives, and some have had their families living there for generations. Yet, because they're not Nords, it's somehow less 'their' home than the Nords (nevermind that Nords wiped out two native populations to move in themselves) It especially leaves a sour taste on how one of the first things the Stormcloaks do is invade Whiterun. Whiterun, for the most part, was staying pretty neutral. They just wanted to stay out of it. Yet, because the Stormcloaks refuse to accept neutrality, and are so "You're with us or against us", that they invaded a neutral city (forcing them to join the Imperials as a result) and attempted to depose one of the better Jarls in Skyrim.
Then, there's Ulfric himself and his 'duel' with the former High King. First off, the High King himself, barely an advlt and just past his teens /adored/ Ulfric - Ulfric was his hero, he looked up to him all the time. According to his mother, if Ulfric had asked, he would have joined Ulfric's rebellion in a heartbeat, or even stepped down. Despite this, Ulfric challenges him to a formal duel, using an archaic custom, and then, having challenged him to a duel with swords.. uses a Shout. That's pretty much bringing a gun to a knife fight. Nords take honor pretty seriously, and tend to dislike magic and think it cowardly, weak, and unpleasant. I would imagine that taking what was supposed to be a formal duel, with swords, and using what is essentially magic to 'cheat' is very, very underhanded - practically cowardly. The only reason I can imagine he managed to avoid the negative PR from this is because people view Shouting as a legendary, manly, Nordic thing to do - despite this, at least half of the population of Skyrim think he's a murderer anyway. Some would think he's a murderer even if he had fought fair, as using some ancient, archaic law to be allowed kill your king, who stood absolutely no chance against you in a fight, and take his place, is a pretty scummy thing to do.
Finally, there was the 'peace conference' which is what finally made me take a side. When I was doing the main plot, I had joined the imperials, but hadn't really done much for them. Then.. I got to the peace conference. All I wanted was to have a temporary, very short term ceasefire, so I could save the frigging world. However.. that doesn't happen. Ulfric swaggers in, like he owns the place, and starts making demands. He has to be given a few cities before he'll agree to a ceasefire.
Hell. No.
Ulfric does not get to hold the ENTIRE WORLD HOSTAGE for his petty, selfish little war. After a while, I couldn't bear any more, loaded up, finished the entire civil war questline, just to kill him. Before that point, I hadn't really known much about how scummy Ulfric acts in general, so I was a bit more neutral, but [censored] did that make me hate him. Now, I hate him enough to write such long-winded rants as this.