Arena: prison
Daggerfall: cave
Morrowind: boat (can be assumed you were a prisoner, but I don't think the game ever clearly stated you were)
Oblivion: prison
Out of four of the games only the first and fourth truly started in a prison, so I wouldn't really say that it's a tradition.
Second: Has it been announced that we're starting in a prison?