Most of my characters are human male, but I enjoy the occasional venture with Argonian, Kajiit, or female characters, just for the challenge of a different perspective than my own. As far as I can see, there's no real reason NOT to play a different six of character, but not that much reason to do so either, other than for some sort of "cheap thrill" of watching an under-dressed character in 3rd person perspective (and which is pointless if you play in 1PP). Since both genders play the game, it would make little sense to limit you. (Witcher, for instance, gives NO character starting options, and is a poorer RPG because of it, where Skyrim gives no MEANINGFUL starting options, which is marginally better).
I like that the game allows characters of either gender with no real penalties, but the lack of distinct body types and the stereotypical "female fashion model warrior" build makes it look kind of silly. If the warriors of BOTH genders looked like warrriors, the shopkeepers like shopkeepers, and the starving beggars like starving beggars, I'd be a lot happier with it.