Ok, now I did run into the guards without upper body armor, and like you said, reloading zone fixes it. It happened after I removed/added some mods and rebuild my bashed patch.
On another note, I am just level 3 with a 16 athletics, and I can't figure out how to survive at all. Since walking is very slow and running isn't that much better, it takes forever to move in a dungeon, and I am always at 0 fatigue when I stumble across baddies, which means I start the fight in a bad way.
If I wait (a loooong time) before starting the fight, that helps for the first guy, but not the second.... and its boring.
Has anyone ever gotten a character to the point that it isn't so painful to play?
1. I think the shirtless guards are a result of adding/removing mods and rebulding patch. I have hit on this last night that is seems to be a problem. Something to do with ROM scripts I think. I have changed my bashed patch to NOT import anything from ooo.esm and ooo.esp except graphics as ROM should be the mod to handle all that. Will see if it makes a difference. Could also be a case of waiting out the respawn interval for ROM leveled lists. Your guess is as good as mine right now.
2. The fatigue is a killer. It heleped when I got to athletics 25. To get it faster that is the only skill I trained. I am level 7 right now with athletics at 31. I can play well as only wearing light armor. Heavy armor is slow speed. Will leave that until I am way higher level.
ONly thing you can do is carry only what you need. Restore fatigue potions svck unless you make them yourself as they will have a longer duration. At low level only fight low enemies like goblins. Others run away from and lead to guards if you can. Unless you have a companion, reduce spawn rates to vanilla if using MMM.
See if you can find any magical items that boos speed/agility or boost athletics. Took me 23 hours to get to level 7.
What I did earlier was to wear fur boots and greaves and steel cuirass. Helped in speed and leveled light and heavy armor from fights. Block with light shield or weapon in a fight to help block skill and raise endurance. Use shield spell if you have it or shield potions. Enchant clothes with shield spell if you can find a way.
Frustrating as hell but I seem to have some satisfaction of doing it the hard way. If this is making the game not fun I would say dump ROM. Don't substitute with TIE as that is just as hard. These are I think for a specific niche of player. If you can hang in there I think it is worth it.
And yeah, don't add or remove any mods once you got the game going

Right now my leveling seems a little off, trying to see a way to get it reset or something. I remember something in the doc about visiting The Copius Coinpurse as something happens when you do with ROM and its scripts.
I think your best bet right now is some companion (or two) but that will keep you from leveling skills as they will take the brunt of the work. Get healing potions (expensive to buy). Making them at low level svcks with low alchemy but better thasn nothing. It may be too late but next time I start a game with ROM I will make sure I have restoration as a major skill

That is all for now that I have.