I was using the unconsious worshippers at peryte's shrine to train my blade skill and after getting it up to around 65 I stopped and decided to go to the Imperial prison and get arrested because all those unreported assualts can glitch your game. So I went in the prison and stole a quil of the Jailor's desk and paid my fine. When I reapeared outside the prison two of the unconsious worshippers were outside and in front of the door. When I went back in the prison they were there too. They only appear here though, I tried going to the Arena they didn't appear there. Just incase you don't know what I'm talking about.
http://www.uesp.net/wiki/Oblivion:Peryite

When you get arrested and someone\thing is still stalking you they will be placed just outside the prison when you are released at times (this is how to fix being unable to rest\sleep when someone is chasing you and the game does not reset properly) -- so that is probably what is happening - the worshippers you were hitting have entered combat with you and will continue to follow you until they are killed (are you able to wait\sleep ) - you might need to do the shrine quest and then kill them to get it fixed (or reset them so they are not in combat - not sure if that will mess up the shrine quest though !)

Its a glitch sometimes certain people appear for no reason. I had that happen with the 3 people in the fighters guild if I got arrested and went to jail they usually appear outside after I got out of jail.

I've done this glitch before and this is the first time they appeared at the prison. I can still wait, fast travel, and sleep. I've seen the same thing happen to members of the fighters guild.