I apologise if this has already been mentioned. On my way into the dungeon from the Jarl's throne room in Whiterun, I noticed that the first cell on the right had a dead prisoner in it. So naturally, I broke into the cell to see if he had any loot. He didn't but he was lying on top of a locked grate which I then picked through and dropped down into a tunnel system. Following it through I entered a larger space with an iron gate in front and a set of stairs to my right. Immediately up the stairs was a window, beside which was the Prisoner Belongings chest. Both the locked gate and the hole you drop into if you follow the second floor round lead to a door to the guard barracks.
While this doesn't get you out of the city, it does get you out of the jail and away from Dragonsreach where most of the guards are concentrated. I remember seeing what appeared to be a way out of the city above a drop on the South edge near to Chillfurrow Farm. My inital thoughts were that the drainage ditch from the prison would in some way connect with this and give you a perfect escape. This doesn't appear to be the case but I'll continue looking.
Unless someone else has already found it. =p