Is there any other way to govern? The one with the armies wins. Yes, you need good systems of governance and diplomacy and bureaucracy, too, but you can't get those unless someone's got big guns and the will to use them. The Roman Empire wasn't built in a day. Latium spent decades pummeling its neighbors and robbing them until they had enough control over the Italian peninsula, and then the Mediterranean, that they could go on merrily absorbing territory.
The risk is that you have too many people around who think they're strong (e.g. the sons of Alexander the Great). That's exactly why Ulfric's thu'um during the duel was necessary. If he had just beaten Torygg at swordplay, every Nord swilling mead in the taverns would think he could have done it. The fact that he has a power others don't (even if because of their own laziness) is exactly why he stands a better chance at leading Skyrim than that waste of space Elisif.
I do agree that Ulfric does not appear to be a good politician. I won't support him as gladly with a character who doesn't want to be his right hand and take care of that end of things for him.
I'm not suggesting, even momentarily, that nations are
let's hold hands and sing kumbaya'd into being, or held together by gumdrops and rainbows. Nations are unified force, and in turn, are unified by force. I'm saying that a system of governance which actively encourages its nobility to slaughter each other for self-advancement is self-destructive. Alvor even notes this, should you follow Hadvar out of Helgen - that once the Jarls start killing each other, Skyrim has returned to "the bad old days."
That Ulfric can Shout says nothing of his ability to rule, and is in
no way whatsoever a better claim to the throne than Torygg's, or even Elisif's. If kingship were a series of duels, then sure, the Voice was be very useful for cheating his way through them. Torygg's strength seems to have been recognising the skills in others, hence his very capable court in Solitude, which appears to be a very well governed city. Elisif has inherited this court, is willing to listen, and shows great empathy towards her people. I'd rather a High Queen Elisif, backed by the Empire and surrounded by capable advisers, than High King Ulfric.