Unarmed is tricky, I've heard that it's governed by one handed skill and is considered dual wielding, which makes sense, but if this is true or not, I don't know. A stealth character will always get the biggest benefits by going bow and dagger style, wearing full light armor, due to the assassin perk that allows you to back-stab for x15 damage (combine it with a pair of gloves that any assassin character should quickly come by, and that will effectively be x30 damage).
There is a perk for heavy armor that allows a character to deal much more damage un-armed, but wearing any heavy armor (and thus preventing oneself from wearing full light armor) prevents you from making use of certain light armor perks that are quite beneficiary. That means that in the long run, light armor wont pay off. Instead, you could focus on heavy armor then, by getting the top perks and getting good at sneaking, you could eliminate most of the drawbacks from using heavy armor as a stealth character.
If you chose to do this, I suggest joining the dark brother hood as fast as possible. The gloves that are part of the guild armor have an enchantment useful for backstabbing.
Also, I suggest getting good at enchanting and booking the carriage from Whiterun stables to Riften and getting down into the ratway. There is a guy down there who drops a pair of gauntlets (light armor, sadly) with an enchantment that does not require recharging, which causes the wearer of the gauntlets to deal 10 additional damage when fighting unarmed. Disenchanting these gauntlets will enable you to enchant a pair of heavy gauntlets. I suggest doing the same with the dark brotherhood gauntlets and putting both enhantments on a pair of heavy gauntlets. If unarmed strikes count as dual wielding one-handed weapons, this should secure you a nice damage output, but my guess is, that it will still be second to most of the good weapons in the game. Backstabbing with a dagger and certain spells seem to be the real killers.
Otherwise, it really sounds like a cool idea, mainly because a lot of the shouts actually seem to support this concept.
If this doesn't work out for you, progressing along the main quest-line for a while should get you access to an akaviri katana (or a blade's sword), which is a one-handed weapon in this game, and thus also good for backstabbing. Not as good as a dagger though. This sticks to the ninja concept somewhat... (and yes, I say that at the risk of getting flamed by Japan-fans

).
Just a note: The kahjiit aren't actually Far-east-asian in culture. They are more like India. They are gypsies mostly, as far as I understand, and their country is mostly jungle and desert. There was a samurai race in Tamriel, but they are extinct, namely the akaviri.
For a stealth-assassin character build, kajhiit is a good race, but I strongly suggest focussing on dual-wielding with a dagger in one hand and a sword in the other, and using the bow as a primary weapon until your sneak skill is very high.