Off the top of my head, swords are fast but have a low chance to stagger on a power attack, while maces are slow but stagger well. Axes are somewhere in between - but how happy a medium they are is really your call. Perks-wise, Bladesman (the sword perk) seems to be unpopular, mainly because the extra damage is calculated from base weapon damage, and isn't affected by extra damage from smithing, skill level, or the Armsman perk.
That said, don't let numbers rule you - I played an archer/swordsman quite happily for many levels, with no regrets.
Along with Areial's footwork recommendation, remember you can also sprint-dodge (i.e., hold down the sprint key and strafe left or right). Great for those two-handed power attacks from opponents against whom bashing doesn't seem to work. A disclaimer, though - I only sprint-dodged with a light armor character, so I'm not sure how effective it would be with heavy armor.