One thing that slightly bothers me about Skyrim is that hardly any of the races fit in with the province and are hard to role-play. I made a wood-elf and made his backstory so he was born in Skyrim and stuff but it got hard because Hadvar at the beginning is like "We'll return your remains to Valenwood" when I'm not even from Valenwood. And a bunch of other NPC's kept going on about how I must be from Valenwood and stuff. It broke the immersion and made it hard to role-play an exotic race in Skyrim. I bet Khajiit must be even harder. It seems that all races except for the human races don't fit in Skyrim. My main character is a Breton and it's a little easier since High Rock is right beside Skyrim. In Oblivion, Cyrodil was a very multi-cultural place with lots of different races and it was bordered by many other provinces while Skyrim seems to be almost all Nords and the game seems to favour them since it is Skyrim afterall and home to the Nords.
I think you're looking at this from the wrong perspective.
Even if your Bosmer was born in Skyrim, he likely has relatives in Valenwood and would want his body to be delivered to the native land of his Race/culture/beliefs/family.
Or, just ignore what people say. If you really want to roleplay, just create it in your head and ignore the few small details that don't fit with your ideas.
Roleplaying is more about
you, than about the game.
In Morrowind everyone hates you and calls you Outlander, no matter what Race you pick. But that never stopped me from role-playing a Dunmer that was "Born on Vvardenfell, was the nephew of Vedam Dren, and hated foreigners for trying to change the culture of my people."
I'm playing a Khajiit in Skyrim, and I find it works great. Being a lone adventurer in a foreign land is fun.
If you don't like that, play a Nord.