Realistically there would be some kind of magickal binding of the creature to the earthly plane, and also some kind of charm or command over the creature attached to the caster, ne? Summons disappear after the caster dies, for example. Hand-waving time!
While this is true, I do agree with Pjstaab that player should really be affected by anything.
It would be cool if we'd be able to use a magical "vanish" to...."wash off" the magical summon
(aka dispel it directly, but that it would require like 100% dispel)
edit: or maybe the summons should have a weakness to dispel magic and it would deal a much higher multiplied damage to the summon than other magical spells.
(sorry, brainstorming over what's not possible
)