It seems more legitimate to vote for someone who directly placed you in charge of a powerful hold with his army pressed at your back, than vote in a free way. This is why secret ballots are so important, no bully tactics to influence votes.
Even with secret ballots, it wouldn't make a lick of difference. Ulfric would be High King without intervention on behalf of the Dragonborn.
Suppose the Moot was allowed to be held immediately after Torygg's death, and Ulfric lost. He'd likely immediately "challenge" (read: Murder) the duly elected High-King, causing another Moot to be held, and would likely continue to do until he was elected.