I played around with the console and set my endurance to 150 (I'm using the mcp attribute uncap, so all points above 100 aren't treated as fortifications). When I ran around my fatigue loss was lessened incredibly. I reset my endurance and set my athletics to 150 (I was also using mcp skill uncap), it barely came close to the fatigue loss reduction (though I did run faster).

I was never aware that endurance played a much bigger role than athletics when it came to reducing fatigue loss.

It's not just Endurance. A character's maximum fatigue is the sum of their Strength, Willpower, Agility, and Endurance.

I'm not referring to max fatigue, but reduction in fatigue loss.