Am I missing something or what? If a man and a woman could create X amount of offspring, how would the offspring make more offspring? Wouldn't they all be siblings and create "freaks of nature" babies? Sorry if there's some obvious answer that I'm overlooking, but this is really bugging me now.
Sure they could. You'd end up with a population that had a big "founder effect".
We don't know of cases where an entire human population was due to a single pair of founders. We do know of cases where the number of founders (either migrants or disaster survivors) was no more than 17, and of cases where the number of fathers (due to religious polygamy) was no more than 2.
In that situation, deleterious dominant genes get eliminated immediately, and deleterious recessives get established at low frequencies. But you don't get a population of freaks or an unsurvivably large number of non-viable pregnancies.