First off, violence and spectacle are not what make a combat system fun. A psychological test using Half-Life 2 replaced the gun noises, guns, and blood splatter with all non-violent looking assets. People had nearly as much fun playing the game as when all the normal violence was present. Spectacle such as the “kill moves” is just that, spectacle. Fun to watch but not to play.
What I'd love to see is a wide variation of how each type of weapon handles. Shields should block, weapons should parry. Parrying can result in a melee weapon being blocked, or if more skilled thrown to the side and momentarily presenting an opening. Blocking with a shield might make it hard to throw a weapon out, but logically should also be able to block arrows and magic.
“Staggering” an enemy is a good game mechanic, but again should be about what weapon you're using. A heavy mace or hammer is going to be a lot better at this than a short sword. Blocking a mace is also going to be less effective since it's going to have a lot more force behind it. Alternatively swinging a sword or something lighter and better balanced is just going to be plain faster than a heavy mace.
While a “shield rush” might make sense with a shield, a more “Duke/Spartaaa!” style bootkick might be a better alternate attack when not using a shield (governed by hand to hand, if it exists). A shield bash might shove an enemy over, a solid kick might trip him.
I'd also love to see the length of a melee weapon matter more. A spear (if they exist) should be good at keeping enemies back, perhaps by “staggering” an enemy a tiny bit when getting hit. You don't just ignore being stabbed by a spear after all. I'd love to see a “press and hold” like power attack, trading time and vulnerability for a more powerful attack. And of course I'll plead the need for a dodge until I'm blue in the face. More game mechanics = better game.
Overall message, interesting gameplay mechanics and a good “back and forth” battle first, “visceral” very much a second. We've gotten “the combat needs to be more visceral” before both Morrowind and Oblivion, neither of which worked out fantastically. Make sure the combat is fun by itself, without cool finishing moves, lots of blood and etc. Only after it's fundamentally fun should you add cool spectacle on top of it.