Really, I would assume that the fathers traits wouldn't be ENTIRELY without presence, that would make the idea of mammalian reproduction a complete failure of logic.
This is TES breeding, not real life breeding.
It can be assumed, due to internal and external differences, such as in genetics, Argonians and Khajiit would be incapable of reproducing with Men and Mer.
Currently, there have been no records of Khajiit or argonians successfully producing offspring from mer or men,
Remember that Orcs are "Changed" Mer, though it has been proven that they can have viable offspring with men, and, presumably, Mer.
They're still mer, like the dunmer who were also changed.
It would be impossible, presuming they all share semi-Mammalian reproduction that Argonians and Khajiits could even conceive a child, and given essentially 41 percent of an offspring's traits are dictated by the father, though many recessive traits are often lost during conception, the resulting abomination would be horrendously disfigured by its fathers traits, as they would not be physically compatible with the mothers traits. Such a child would likely die of genetic incompatibility, if it was even born at all.
Really, do not think TES breeding in real life breeding. As far as you should be concerned, genes are not there, and the mother is almost completely dominant,
The main reason the offspring of men and Mer were noted as mostly retaining the mothers traits is because the main difference between the different races is skin and eye color, and that is usually dependent on the mothers. The reason the offspring of a Dunmer father and Altmer mother would look more like an Altmer is beyond skin and eye color, little is different in the facial and proportional body shape of the two. Occasionally such a child may have the red eyes of a Dunmer, or perhaps darker skin, would be because of some sort of Recessive tendency of the mothers traits that makes the fathers visible.
It's just mom is dominant, dad is the 1% at best.
This is, of course, merely a thesis based off of what I learned in Sophmore biology, and a bit of inference based off of known elder scrolls lore.
Again, real life doesn't work here.