From what I understand, turning a skill legendary, gives you back all your perks in that skill, and reverts it back to zero. How does that make sense to do?

It allows you to raise your character to higher level, by letting you raise the skill all over again.

Consider this, for instance: Raise your Smithing and Enchanting to the max, and then make top-notch enchanted gear. Then you won't need to use those skills, so you can make them Legendary, starting them over. Train them back up, and use them to push your character's level higher.

From a roleplaying point of view, it makes no sense at all.

Nor does the name of the function. My Illusion magic is now legendary. I have no freakin idea how to do anything with it. So epic, fear my might.

Roleplaying wise, it would make sense if skills would very slowly reduce themselves if not used for a long period of time, although I can imagine a lot of players hating that, and it kinda has nothing to do with it being legendary, but you get my point. I'd love that kind of realism.

In all honesty, I don't understand why Bethesda didn't simply uncap the skills. If they didn't want them to be too powerful they could have just capped their calculations at 100. Or have them contribute to the level upon level up, but still have them capped at 100. Design and implementation, and it's name was done in the most amateurish level possible. I love you Bethesda, but this is one of those things that....

There is some justification -- for example, I have good penmanship, but if I were to write with my left hand, it would be like my penmanship "skill" had never been "raised." Similarly, a character who uses a dagger is just as good at using a sword because both weapons are governed by the One-Handed skill; if the character had reached 100 One-Handed by using a dagger, you could Legendary the skill and decide that they would now learn how to use axes.