I like to play them both. But i have a confession. Not long ago, before i really gave it a serious try, i kinda looked down on men who preferred to play as female characters. I completely understood why women would do it, but men???? that did seem a little... strange. But now i completely understand the fascination. It's not often you get the chance to be in the shoes of the opposite six. It can be very entertaining and rewarding to see the world through the eyes of a woman and it opens up for new exciting roleplaying possibilities. While i still mostly play as males, i suspect about 30% of my characters is female and it wouldn't surprise me if it's fifty-fifty in a year or two. This also depends on the games however, the experience needs to differ.
Yeah, I was sort-of the same way, I didn't really get why people did it. But it isn't as much as a fascination as it is an "empathization", for me anyway. You sort-of realize that you can play as a male or a female regardless, and still make a good character and make up anything you want. I actually started trying the idea when I was writing a little short-story with a female as the main character. When I first played as a girl in Oblivion I felt really weird and awkward, it just seemed odd and foreign. But eventually I got used to the idea.
I don't do any of the stuff like downloading mods that make the armor more revealing or anything, because I still like my characters to have some self-respect and dignity. And you can easily place a female character in the role of almost any male character. I do it for the sake of variety and the ability to take a few different avenues as female characters, but I still play male ones about 50% of the time. Mostly because female characters just sometimes fit some roles better than others, and same with male ones. For instance, I have 1 character who's royalty, she lives in a castle and her father is the Count of an island. While she sometimes listens to her father's advise about acting more lady-like she usually kicks those values to the side and goes out training with the guards and warriors of the island behind the castle on the training grounds and wields a Silver Claymore. It wouldn't be as interesting if it was a male character because his father wouldn't be telling him to not be doing what he's doing.
In the same aspect, I have a few male characters that are pressured by their families and/or friends to join a faction based on combat and "man-up", but they'd much rather do other things, like studying alchemy or writing articles for the Black Horse Courier.
It all has to do with how far you're willing to go with roleplaying.