Lore wise, they are a large part of the economy on Vvardenfell, since mining eggs is a very profitable industry.
The eggs are a basic food that everyone eats.
That's correct.
As for their interest to players, some egg mines are quest related, but aside from those, I haven't really found anything of particular interest in them beyond perhaps providing ingredients for potions to sell or practice alchemy, since kwama eggs are fairly heavy, I don't really bother using them in potions I actually intend to keep, if I need restore fatigue potions, I prefer lighter ingredients like saltrice. In any case, I generally don't bother exploring egg mines unless I have a quest that requires me to enter one.
I think the main reason the developers put egg mines in the game is to help create the illusion of a living world rather than to be dungeons for players to loot, after all, food is a basic requirement for people to live, so if you want to create a believable world, it's only natural that you'd adress some of the places it comes from.