Sampled Neandertals (from Europe, the Caucasus, and Siberia) certainly had lower effective population size than living humans, but I wonder what the comparison would be between ancient tribes of modern humans and Neandertals in the Near East where admixture presumably took place.
The Genetic Cost of Neanderthal Introgression
Kelley Harris, Rasmus Nielsen
Approximately 2-4% of the human genome is in non-Africans comprised of DNA intro- gressed from Neanderthals. Recent studies have shown that there is a paucity of introgressed DNA around functional regions, presumably caused by selection after introgression. This observation has been suggested to be a possible consequence of the accumulation of a large amount of Dobzhansky-Muller incompatibilities, i.e. epistatic effects between human and Neanderthal specific mutations, since the divergence of humans and Neanderthals approx. 400-600 kya. However, using previously published estimates of inbreeding in Neanderthals, and of the distribution of fitness effects from human protein coding genes, we show that the average Neanderthal would have had at least 40% lower fitness than the average human due to higher levels of inbreeding and an increased mutational load, regardless of the dominance coefficients of new mutations. Using simulations, we show that under the assumption of additive dominance effects, early Neanderthal/human hybrids would have experienced strong negative selection, though not so strong that it would prevent Neanderthal DNA from entering the human population. In fact, the increased mutational load in Neanderthals predicts the observed reduction in Neanderthal introgressed segments around protein coding genes, without any need to invoke epistasis. The simulations also predict that there is a residual Neanderthal derived mutational load in non-African humans, leading to an average fitness reduction of at least 0.5%. Although there has been much previous debate about the effects of the out-of-Africa bottleneck on mutational loads in non-Africans, the significant deleterious effects of Neanderthal introgression have hitherto been left out of this discussion, but might be just as important for understanding fitness differences among human populations. We also show that if deleterious mutations are recessive, the Neanderthal admixture fraction would gradually increase over time due to selection for Neanderthal haplotypes that mask human deleterious mutations in the heterozygous state. This effect of dominance heterosis might partially explain why adaptive introgression appears to be widespread in nature.
The Strength of Selection Against Neanderthal Introgression
Ivan Juric, Simon Aeschbacher, Graham Coop
Hybridization between humans and Neanderthals has resulted in a low level of Neanderthal ancestry scattered across the genomes of many modern-day humans. After hybridization, on average, selection appears to have removed Neanderthal alleles from the human population. Quantifying the strength and causes of this selection against Neanderthal ancestry is key to understanding our relationship to Neanderthals and, more broadly, how populations remain distinct after secondary contact. Here, we develop a novel method for estimating the genome-wide average strength of selection and the density of selected sites using estimates of Neanderthal allele frequency along the genomes of modern-day humans. We confirm that East Asians had somewhat higher initial levels of Neanderthal ancestry than Europeans even after accounting for selection. We find that there are systematically lower levels of initial introgression on the X chromosome, a finding consistent with a strong sex bias in the initial matings between the populations. We find that the bulk of purifying selection against Neanderthal ancestry is best understood as acting on many weakly deleterious alleles. We propose that the majority of these alleles were effectively neutral-and segregating at high frequency-in Neanderthals, but became selected against after entering human populations of much larger effective size. While individually of small effect, these alleles potentially imposed a heavy genetic load on the early-generation human-Neanderthal hybrids. This work suggests that differences in effective population size may play a far more important role in shaping levels of introgression than previously thought.