I'm not a race-buff, but in fact, there is no real lore about mixed races, afaik. Thats one very unrealistic point in The Elder Scrolls. Maybe a human/mer hybrid is not possible, but a Altmer/Bosmer hybrid have to work. There were a few examples, where different races live together. The last one was at the Shivering Isles, I guess. I'm speaking of the sidequest with the amulet and this knight... don't know exactly.
Edit: Thanks for the link Nalion
If that was the case then there would be no such thing as a Breton. The Bretons are half human and half elf, or human and of elf blood.
Also, it was noted that after the fall of the Ayleid Empire and during their extermination, many of them fled to Valenwood, and other places with elven population, where they interbred with the native elven species.