Really? I haven't played an ES game in ages. Was sure they were all male.
The dremora page at uesp states they are male dominated, don't know if that just means only males fight or if there are only male dremora period, but all the dremora you fight on OB are male.
The dragons could be genderless as since they're magical beings they may not need to breed. Or it could be like Reign of Fire where they're all one gender and then there's just one of the opposite gender, Alduin.