Unless they were changed so much that they no longer count as elves.
It completely matters on what you think counts. For example, most people in Tamriel don't consider the Khajiit as elves. Instead, they're betmer, or beastfolk. But we (and knowledgable scholars of Tamriel) understand them to have been descended from early Aldmer. With that knowledge, I would consider them elves (or, at least, elvish relatives). Its worth noting that in Herma-Mora's Skyrim quest, you aren't required to obtain Khajiit blood. Only Orsimer, Falmer, Dunmer, Altmer, and Bosmer blood. He may not consider them close enough relatives to the Dwemer to matter. But then again, like many people in Tamriel, Septimus may not be aware of their kinship with the elves.
Topal the Pilot saw them quite a bit, and they seemed more savage and feral.
I wouldn't trust Topal's accounts too much. Like many real-life explorerers, his perspective as a explorer deeply distorts his observations. Like real-life explorers, anything he came across that was very different from his own culture would have shocked him. Hence many explorer accounts describe natives as savages, when nowadays anthropologists know they clearly weren't. I also recall that Torval mentioned bird men living along the Niben, and is the only source that we know of that mentions such bird men. Its quite possible that he only saw early Nibenean people, dressed in ceremonial feathered dress, and just let his imagination run wild.
They could have been a few extremely early Aldmer settlers who somehow managed to morph into what they are now.
Not settlers, natives. Most lore forum goers now recognize that Tamriel was the native homeland of all the races. Atmora, Altmora, Yokuda, Akavir...these places were settled by the Elhnofey leaving Tamriel in ancient times. But not all Elhnofey left, of course. Some stayed, and became the ancestors of the Nibeneans, Bretons, Bosmer, Khajiit, and
possibly the Argonians. The Nords, Redguards, Altmer...these are groups whose ancestors had once left Tamriel, but later returned to conquer it as a new land.