Maybe because Torygg support an empire that executed those who worship Talos?
And also, I don't like him, either. I'd much rather have someone else be High King, but Torygg was a coward. Talk to him in Sovngarde.
And the reason he claims to be High King is because he killed the old High King in a duel, i.e the way the Nords do it.
So your argument about him NOT being High King are idiotic.
Torygg was one of the best example of a leader we have seen in Skyrim, he handles himself flawlessly in Sovngarde, putting his wife and people before his own life, marching toward his death without hesitation or self-pitty. That is far from "cowardly", but a shining example of bravery..everyone knew, including himself and Ulfric, that he had no chance at all against such a force, and yet he chose death over dishonor.
Way to try to start another CW flame war.
Excuse me, but 1. You're the one who brought it up. 2. You're the one who asked for more details, 3. You're the one who took the hard line of saying the biggest grievance the Empire supports have, Ulfric, isn't negotiable. and 4, you're the one who got personal and started name-calling.