Without two fixed points, time cannot exist, therefore if the Towers should fall, time would cease to exist. If one was to accept a 1:1 correspondence between time and space, all of everything in the Mundus would also cease.
That being said, Nirn came about upon the ordering of Red-Tower, as Ada-Mantia could not singularly hold time and space on its shoulders without Divine aid. This was the beginning, but just as a song, at least not an interesting one, cannot exist with just a tempo created by the space between two notes, keys, chords, and progressions of the two must also exist. In this sense, more towers were also constructed.
If an important note was to be omitted, some measure of the song would be lost, yes? Therefore, one may argue that with the destruction of a Tower, some measure of time, that is, time and space/ the scope of Mundane Reality would also be lost or at least changed significantly.
Of course, an added note, let's say a poorly formed and particularly off-key note may also cause some manner of loss/change within a song. If one were to introduce a Tower improperly, I'd argue a significant measure of reality would change. Creation and destruction are, of course, closely related.
What then would be the worst thing one could do to a song? To simply stop playing it would mean nothing in the long run, as it could just as easily be commenced anew. If one were to kill the performer or even worse, kill the composer and burn the music, there would be nothing tangible left.
After reading Nu-Hatta yet again, I think I finally realize the dangers of White-Gold. It represents the whole of Aurbis within the scope of a single plane(t); the whole being contained wtithin the part.
It is of course, a wheel, modelling the Wheel. If one were to turn the Wheel, the entirety of Creation and Destruction would follow suit, but such an act is impossible by primary action.
A gear or a spool turning a larger object is not impossible though. If one could turn White-Gold, wouldn't Aetherius move, if not by just a small bit? Reversal, Nu-Hatta calls it, discreation.
There is energy that flows around Nirn. It's the energy created by its very existance. When it turns, energy winds around it like a spool winding thread. If someone were exert enough force in the opposite through a smaller spool, a smaller wheel, the energy could be unwound and harnessed in small amounts, but even a small amount of this energy coul fuel impossible things. Even worse, the displacement of energy would naturally be felt in the greater part of the Aurbis. I don't believe it could possibly undo everything, but it could easily unmake Tamriel and Nirn as well, if the energy were to become permanently desyncronized.