The Chimer were what the Dunmer were orginally, before Azura decided to make a slight change after Nerevar's betrayal. They were more or less Altmeri in appearance.
But yes, Mer are a diverse bunch. Of course, there are several human and beast races not shown in the games as well.
Altmer, Bosmer, Chimer who became Dunmer, Dwemer, Orsimer, Falmer, Maormer, Aldmer, Ayleids, Khajiit which might actually be transformed Elves... there are quite enough of them...