I never get where people keep seeing the similarities between the Altmer and the Japanese. Besides that piece of art that depicts them with a katana, I can't see any similarities.
Well, like every Elder Scrolls race, there are connections to many real-life cultures. Like Nords and the Scandinavian cultures, the Altmer just have a lot of Japanese imagery and themes tied to them.
In Morrowind, for example, a description of Summerset Isles goes something along the lines of describing their policy of isolation and not being affected by Imperial commerce and culture because of this (unlike Morrowind, where the Imperial presence is everywhere). This is basically the same policy the Japanese where famous for, until Admiral Perry broke down the gates, as it were.
Another thing from Morrowind comes from the description of http://1.bp.blogspot.com/-P9jnEJZzHrQ/TVn6o6AUc0I/AAAAAAAAAik/LDLm3ZCQDn0/s1600/Morrowind+2011-02-14+21-47-55-59.jpg, which describes it as a derivation of Altmer armor. The similarity to Samurai armor is unmistakable. This was all, of course, before http://chorrol.com/files/62/obliv26B.jpg http://www.axfiles.com/images/ss/skyrim/001.jpg made its debut in Oblivion, and Orcish armor was turned into something more http://farm6.static.flickr.com/5281/5243099611_ae56c47602.jpg-http://images1.wikia.nocookie.net/__cb20111214004959/elderscrolls/images/e/e7/SR-concept-Orc_Armor.jpg.
And yes, there's the fact that they live on an island, and have been depicted in Japanese-like attire, and that they've been pinned as, like the Dunmer, an ancestor-revering people.
Certainly, with Oblivion, Altmer and Orcs turned towards a Tolkien inspiration rather than a Japanese one, but since we really don't have a lot of information on Summerset and its people, we don't have a whole lot to compare them to. Just a lot of Japanese and Tolkien-elf imagery and themes. So when people pop the age-old and exhaust-less question of "What Real-World Culture?" people say Japanese. And I think that's the best correct answer, because "the elves of Tolkien" doesn't really answer the question.