So we know the Dwemer used their collective souls to form the Numidium's golden skin (effectively creating a massive soulgem) with the intent of reaching back before time to the initial brush of Padomay and Anu. And we know (if you believe the love letter) that the Numidium still somehow holds the world under seige until the Fifth Era. So it seems to me that the mystery of the Dwemer isn't "where did they go?" but "where did the Numidium go?"
I got into TES during Morrowind, and I'm a little unclear on the events of Arena and Daggerfall. I know what the Numidium did/didn't do, but I don't know what it's fate was after it's time-shifting, empire-building phase. Did it vanish into time? Is it still locked up somewhere? Am I totally missing some vital piece of information that really explains what happened to the Dwemer?