Declare a variable in a quest script (quest called 'YourQuest', variable called 'Var' in the following), then use this as spell script
float distref mebegin scripteffectstartset me to getselfset dist to me.getdistance player if YourQuest.Var == 0 set YourQuest.var to dist elseif dist < YourQuest.Var set YourQuest.Var to dist endifend
That
should work. You need to set the quest variable back to 0 after the message has been displayed that tells the player how close the nearest enemy is. Or even better, start the quest only when the spell is active and stop it directly afterwards. That should reset all variables each time automatically.