As I understand it, you have to
hold the scroll and
speak the words at the same time.
And when you do that, the scroll sort of vaporizes itself, or burns to ashes or something, in order to release the magic inside.
This is pure interpretation, of course, but it answers your questions satisfyingly. The scroll needs to be destroyed for the magic to be released, and the words on it are the "activation code" for that, so that anyone can use it, even a stupid warrior (as long as he has read his "Daedric ABC for Barbarians").
[Wild speculation:]
In this case it also makes sense that "Woe upon you" is used so often. It would be only natural that Destruction scrolls would be the type that has been used for the longest time throughout history, since they've probably been created with the battlefield in mind in the first place. And maybe "Woe upon you" simply was the easiest one to remember, or to speak during battle, or something like that, so that other phrases like "I'ma hurt ya" simply didn't prevail (hey, many internet users use the same password for every website, why would warriors in a fantasy setting be less lazy?).
As time passed, other types of scrolls had been made as well, but there was no good reason to make up new activation codes - everyone was accustomed to "Woe upon you". Some priests, or mages with a sense for poetry, still try to use other phrases from time to time, but to the common scroll-user, that's more of an annoyance rather than a welcome change.