Lemme pull up all relevant dialog that I found in the construction set.
I am a monk. In mind and spirit, I am a student of the hidden mysteries, of the unseen world that lies behind the objects and ideas of everyday life. In body, I am a student of the ancient martial arts, patterned on the 'Rain-of-Sand' fighting styles of Elsweyr. I pursue hand-to-hand and unarmored self-defense, because what you don't have, you cannot lose. I also train with the blunt weapons called staffs, because even the poorest peasant can own a staff.
Under "Martial Arts"
The unarmed, unarmored martial arts traditions of the Marshmerrow, Salt Rice, and Golden Reed societies of the Dissident Priest, patterned on the 'Rain-of-Sand' fighting styles of Elsweyr, are slow to be adopted in the Empire, associated as they are with the ascetic renunciation of worldly wealth and material goods, and the rigid disciplines and mystical philosophies so alien to the Imperial West.
Under "unarmored"
Monks dedicated to the ascetic philosophies of Elsweyr have long cultivated the unarmored martial arts traditions called the 'Rain-of-Sand' fighting styles. The Mages Guild and other societies have had some success adapting these styles for self-defense training of wizards and other armor-averse vocations. Students of the unarmored styles wear little or no armor, and count on evasion and deflection to avoid injury.
So yeah, the khajiit are really the only hand-to-hand fighters out there, other than Drunken Nordic Tavern style