I get that this is likely for role-playing quirks - a warrior shouldn't have access, etc - but if one wanted to role-play as a wizard, what happens if they want to be someone who knew magic prior to the incarceration? Wouldn't it be easier to just avoid the spells than to ruin it for someone who specifically wants to have them from the start?
Actually if I was playing a warrior, of course it would also be a bit odd, but this is more relevant to playing a mage, for me.
It's more interesting for me to play a beginning apprentice mage who knows no magic at all and learns spells from tomes or training. I will appreciate every tome I find all the more. I think it would be great as others suggested if we find a spell tome in the cave where we start the game.
Otherwise, just starting out automatically knowing spells, especially something really cool like Clairvoyance, takes away part of the mystery of studying the arcane arts. What kind of creature are we that we are born with this sort of knowledge.
Some people like to role-play characters that already have some knowledge of their profession, but I prefer to roleplay complete novices, who are learning from day one through adventuring, speaking with wise NPCs, etc.
As for special racial abilities, these are part of your heritage from whatever race you are, nothing wrong with starting off with these.