I like this idea. Some perks might be available to more than 1 race but not all races. I can see both Nords and Orcs having access to Berserk type perks.
I'd say between 10-25% of perks should be race-limited. If not every Bosmer is going to have the same perks there need to be a decent few to choose from.
This is key. There should be some perks that are only available to a specific race. There should be some that are only available to a specific type of race (some for "warrior" races, some for "magical" races, that sort of thing) and there should be enough of them available to choose from that it will add to differentiation not only between the races, but between individual characters.
For instance, my Bosmer hunter could take the perk that grants him a higher level cap on Beast Tongue, so he's not still stuck only being able to command rats and mudcrabs when he's level 20. My Bosmer thief probably isn't going to be much interested in that, but he could take the perk that grants him a bonus on negotiating with merchants and/or fences. And no other race is going to have access to the Beast Tongue one (obviously), though, say, Imperials would also have access to the negotiation one.
The only problem I see with this idea is that I'm going to be disappointed if it's
not in the game. It's really such an obviously good idea, and wouldn't be at all hard to implement.