fDistanceInteriorReactCombat is the distance at which a hostile actor (NPC or creature) will attack, but only if the actor has detected the player.
FDistanceExteriorReactCombat determines the same for wilderness encounters.
If they're still saying things like "Damn rats. etc.", it means they haven't detected your presence yet, so changing those settings will have no effect. I once got very close to 2 bandits, sneak killed one and the other just stood looked at the dead body and said "Must have been the wind.", meaning he didn't detect me even though I as right in front of him (and no, I wasn't using chameleon or invisibility, just sneak).
Check out this http://cs.elderscrolls.com/constwiki/index.php/Category:Detection It has a lot of detail on Detection and what influences it.