Well he isn't the only Dwemer left any more ?
He is. Every dwemer, including the ones in Hammerfell, were absorbed onto the Numidium as its golden skin. The last dwemer, somehow, wasn't. And Vivec couldn't sense the dwemer anymore throughout time, space, and everywhere else. This could mean a few things; there was a slight oversight of the devs with Vivec's dialog, Vivec ignored him, or Yagrum is something else now.
The point is, the dwemer are no more. They tried to do what the altmer constant dream about, and whether it succeeded or not is still up in the air. To clarify, the dwemer, pretty much, tried going backwards by infusing their entire race into the Numidium to become a god once again; be there during the first strokes of Padomay and Anu. Vivec believes, other than they didn't know what love was, that their desire to go back is fundamentally flawed, and maintains that if one is to reach the final gradient, one has to go forward. This sort of thinking is completely against what other mer, especially the altmer, believe. And to make himself more blasphemous towards other mer, he maintains the creation of the world is Shor's divine mercy, as opposed to a trick to weaken everyone, which practically all altmer hold true.