As has been said, it's an Elder Scrolls tradition, and I'd say it has some role-playing and narrative advantage, for one thing the prison beginning justifies your character being in a certain place at a certain time while still leaving things up to your imagination, because it doesn't tell you why you're in prison, you just know your there, and are free to decide why. It can also justify you starting the game without any sort of experience or equipment. If you started the game as a farmer, then that automatically forces a background on you, if you start the game as a mercenary, it's the same and the game also would be expected to give you at least basic starting equipent. Now, I guess the game could also achieve a similar effect by having you held prisoner by bandits, thus getting a similar feel to the prison start without the story assuming you're accused of some crime, that's one option Bethesda could consider if they want to find alternatives to being in prison, but for now, being a convicted criminal (Rightly convicted or otherwise is up to you.) seems to be something that's here to stay.
If you want a back story that doesn't involve being a criminal, yeah I would say so.
I could think of a number of backstories that could explain your character being in prison without you being a criminal, at the very least, you don't have to actually be guilty of anything, maybe your character was wrongly accussed or framed, that way, you can explain being in prison while still having a background that doesn't involve commiting any crimes.