Instead of having to go to your folders to delete it every time?

Nope, they have to be manually deleted. You can usually leave up to 25 save slots in the folder without encountering any problems. I generally overwrite the same save 4 or 5 times before creating a new one, so 20 to 25 slots will carry me quite a while.

They can also be deleted from Wrye Mash. It isn't exactly quicker, but it's much easier to see exactly which file you're deleting.