What's up with the castle since the guards keep climbing and strolling around the walls even when all guard posts have been removed???

Trying to block access to the walls don't work, they just spawn up there and are locked.

Seems like any guards at the castle have pre placed patrol points. I frequently see them standing at the star points, one guard always hangs out by the radio station, and the spot where the west wall is broken - which since I have "repaired" with concrete blocks and a wooden floors http://images.akamai.steamusercontent.com/ugc/367408984873917773/A6D069CABB7EC9289BFFC1FF42FA6FFF19FDEEC3/, the guard still stands at the pre placed patrol spot and is standing shin deep in floor.