How do I set the condition on Mod Incoming Spell Magnitude so that it only works on incoming effects by a source other than the perk owner?

(It is NOT the same as Weakness to Magicka because Mod Incoming Spell Magnitude will also affect beneficial spell effects, so I can't use that).

I don't know why this shouldn't be stupidly simple, but I have been unable to set a condition on this perk so that it will only work if the perk owner did NOT cast the spell.

The nearest workaround I have managed for a 50% magnifier using this perk is: (simple example)

Perk name "vuln"

Mod Incoming Spell Magnitude x1.5

Mod Spell Magnitude 0.67, Target condition HasPerk "vuln" == 1.0