Mer and men descend from a common race, which along with the Hist were the first sentient(?) races.
Then along come the Aldmer which branched out into the Altmer, Bosmer, Chimer, Falmer, Ayleid and Dwemer races.
The Chimer were turned into Dunmer by Azura
the Dwemer, Falmer and Ayleids disappeared
and some Aldmer were transformed into Orsimer/Orcs when Trinimac was reborn as Malacath.
Men on the other hand developed separately on separate continents:
Nords (Atmora), Redguards (Yokuda), Nedes and Kothringi (Tamriel) and possibly the Tsaesci (Akavir).
The Bretons were spawned from the interbreeding of indigenous humans (Nedic/Kothringi) and Aldmer.
The Imperials descended from the interbreeding of Nords and Nedes/Kothringi.
Did I get it right?