Wisps typically don't take physical damage. You'll need weapons that do fire/ice/shock/health leaching/etc... Basically need to kill em with magic.
Yeah you are right, I just managed to kill her and I had to do it with magic they don't seem to mind physical damage much, I fired up my noob fireball in the left and lightning in the right while summoning fire antronachs, while not great at magic it was the only way I could find to kill her! She's tough, I wish I read about her first!
Btw the view from up there is amazing, that Aururas or whatever you call it.. effect with the swirling colors in the sky is incredible, probably the most beautiful effect I have seen in any game! Then you have the wind swirling the snowdrifts off the peaks... wow., the attention to detail in Skyrim has me constantly staring in wonder...