No, there's no lore reason, it's a game mechanics reason - or more specifically, a budget consideration. They spent all their voice acting money on Sean Bean (and 10 lines by Patrick Stewart) and had to cut back on unique voices for the rest of the races.
All Elves of the same gender share a voice (High Elf)
Orcs and Nords of the same gender share a voice (Nord)
Khajiit and Argonians of the same gender share a voice (Argonian)
Imperial and Breton females share a voice (Imperial), but the males are distinct.
Redguards are the only race with a unique voice for both genders.
That is indeed true, Bethesda often used the the same voice actors for multiple races to save on budget. In this case, though, while there might not be any particular lore reason for Dunmer sounding different in Oblivion, there is a justification (though just for the voice being different, not for it being the same as other elves.) because not everyone of the same race would sound the same, in real life, listening to the voice of two people of the same race, you can probably tell they're different people even without looking at their faces. If Bethesda had the budget to accurately portray this, Dunmer of the same six would not all sound the same either. It may also be that it's mentioned that in Morrowind, non-native Dunmer can be identified by their western accent, though I doubt an accent can account for the raspy voice of Morrowind's Dunmer whereas in Oblivion they sound entirely different, maybe one could blame it on airborne ash from red mountain, but that doesn't explain why females sound the same in both games, or why non-Dunmer who have spent their entire lives on Vvardenfell don't have raspy voices, though Bethesda not being able to afford giving them unique voices might. Ultimately, though, the reason for the change of voice is for budget reasons, not lore reasons.
That video pisses me off for more reasons than just the voice. It sounded like a natural conversation, for one thing, not ME HATE MUDCRAB, YES ME AM HATE THAT CREATURE YOU MENTION TOO, GOODBYE, BYE, HELLO, ME AM HATE MUDCRAB.
That's probably because it was likely specifically written for that video to show off the game's NPC conversations, and stop acting stupid. The NPC conversations in Oblivion might not be good, but the responses are at least usually grammatically correct, if you think you're acting clever, you're really doing the exact opposite.