I'm pretty sure that's the AI lagging. Reducing the number of ESPs you're using may make them turn hostile faster. My last install did it all the time, but my current one with the same machine doesn't.
Well this is within a game where the number of mods have stayed the same from the start, although I do typically play with a large number of mods. But its not just this play-through; usually I play long games (i.e. stick with one character through a lot of levels/quests) and this delay in attacking, where you can walk up and start a conversation before they attack, has always happened to me in the later stages of the game's progression, not so much or at all during, say, the first 1-20 levels or so. I usually have my mod build pretty much finalized before I start playing a new game, so the number of mods typically changes very little during a play-through. So maybe the direct cause of the lag is the size of a saved game. (Which of course will be made larger more quickly by a large mod build.)
So if I understand correctly we have two theories in this thread.
1.High personality/speechcraft/reputation.
2.Large save file size/mod build.