Actually I wasn't imaging that scenario at all, oddly enough. I was in fact imagining my idea of a person who has spent decades honing his hand-eye coordination, agility, stamina, and general hand-to-hand combat, facing a swordfighter with an equally impressive resume. I am still quite sure that your estimate is way off. Again I do not claim to know a whole lot about masters of swordplay or hand-to-hand fighting, but then again I can't imagine you know much more than I do.
Well, I obviously couldn't consider myself a martial art master but I did study it for 8 years, and I realise that a martal artists defense against someone with a cutting weapon like a katana is rather flimsy, most of our blocks involve taking some of the force of the hit while simultaneously moving the hand/leg away from our body, with a blade you could lose an arm doing that. Any attack a martial artist would do would leave him insanely vulnerable, a simple side kick would leave your leg exposed for just a half second, not easily exploitable by a martial artist, but a swordsman could easily cut and cripple the person.
Arguments?