If you delete the ESP, the files will not effect your game anymore. But you might as well delete them if they aren't being used though, to save space.
Keep in mind that some mods use the same files (resources used by more than one mod, for example) so deleting them would cause problems.
http://www.mentalelfz.com/tesfiles.html has the ability to pull all files associated with an ESP, making for an easy uninstall.
But, if your problem is just some GMSTs, why don't you just clean the mod? It takes one second with a tool like http://planetelderscrolls.gamespy.com/View.php?view=utilities.detail&id=34 or http://planetelderscrolls.gamespy.com/View.php?view=other.detail&id=52.
Thanks
Im sceptical about using TESTool or any program.
I use TESTool to tick mods, yes, but not clean.
Are uncleaned entries which aren't evil acceptable? Are mod conflicts alright too?
I just get rid of mods with "evil" GMST entries. But I ignore other unclean entries. Is this safe?
There are some entires with "s" and "f" at the beginning.
But aren't on the evil entries list:
http://www.mwmythicmods.com/evilgmsts.htm
is GSMT and other unclean entries allowable?
I have heaps of conflicts, but less uncleaned entries. I don't see conflicts harmful, since there's a chance that one entry will trigger, and the other is disregarded.