I am unsure of whether this is a bug or not, but The bandits were just not moving around or doing anything/fighting back. I am getting rid of Sands of Time just in case there is a problem with it.

Any other possible ideas? If this is a mod issue, then please move this thread there.