The only reason I'd suggest "safe" travel is if you're going to Daggerfall; if you travel recklessly and end up there at night, you're going to get some dude VEEEEEEENGEAGEing all up in your shizz.
If I recall, in any walled city, if you arrive there at night you can also be attacked by a group of rogues/thieves/assassins at the city gates.
But I don't remember if it happens when arriving at a walled city at night, of it happens when you're released from jail at night.
I just remember being butchered by thugs when I got out of jail. It's even more dangerous when you get released from jail with 1 HP.