My first encounter worked fine. I redesigned the level portion the encounter takes place in so I deleted the original three lvlbandit*s I had placed and now the new ones won't attack unless I try to initiate a conversation with them. I checked their faction and it was set to none so I set it to BanditFaction and still they won't attack. They all have the masterambushscript.
They're set up with simple two point patrols and they navigate those with no problem.
My follower can go anywhere I go in the level as well.
I've actually encountered this bug with retail bandit camps as well but those are random and unrepeatable whereas this problem in my custom dungeon is persistent.