Direction of gene flow (to or from Neanderthals, or ...?)

In my first post on Green et al. (2010) I bring up a couple of red flags to the tale of Neandertal admixture that has appeared in the media:

1. The uniformity of alleged Neandertal admixture outside Africa is suspect, given that Neandertals were a West Eurasian species
2. The observed pattern of Neandertals being closer to non-Africans than to Africans can be well explained by structure in Africa itself, a point which the authors concede (as their Scenario 4), but which has received zero play in the media which -as usual- have jumped on the more easily digestible ("caveman sex") explanation.

Now I want to address another major red flag that I hinted at before, namely the direction of gene flow, if gene flow did in fact occur between modern humans and Neandertals.

The authors reject that gene flow into the Neandertals took place; I immediately got suspicious of this claim because it is not consistent with what we know from historical cases of contact.

Whether it is Europeans meeting Native Americans, Yayoi meeting Jomon, or Bantu farmers meeting Pygmy hunters, the story is always the same: the dominant intrusive population ends up with admixture from the native one, but admixture goes both ways.

If modern humans and Neandertals interbred, then there is absolutely no reason to think that "we" got Neandertal genes, but "they" didn't get ours. To think that requires extra assumptions, e.g., that modern-Neandertal kids were shunned by Neandertal societies, i.e., that Neandertals were a sort of prehistoric Samaritans that practiced strict endogamy. I find that hard to believe.

There is an alternative explanation for why no modern admixture in Neandertals would be detected, namely the early age of the bones tested in the paper, which are from Europe and date from the Early Upper Paleolithic, when modern humans had not yet arrived in Europe. It is worthwhile to consider this explanation:

If admixture occurred primarily in Europe itself, then we would expect Neandertal admixture to be higher in modern Europeans or Caucasoids. But we don't see that at all, so, unless we postulate that Papua New Guinean and Chinese ancestors reached their final destination with a detour through Europe after 40ky or so, we can safely conclude that Neandertal-modern admixture was earlier and occurred in the Middle East.

So, we have a population of moderns and Neandertals mixing in the Near East. According to the authors' scenario, episodes of admixture between the two species gave "us" enough Neandertal admixture that was carried by our (modern) ancestors throughout the world, creating a fairly uniform (but small) component of "Neandertal" ancestry.

But, why wouldn't the (modern) admixture in Near Eastern Neandertals also be carried a couple of thousand km into Europe? In short: we are supposed to think that Neandertal admixture in modern humans made it to the ends of the earth, but modern admixture in Neandertals could not move a short distance from West Asia into Europe. The explanation of pristine European Neandertals presented in the paper just doesn't fly in the context of the authors' broad model.

It is also worthwhile to consider the actual genetic argument for no modern-to-Neandertal introgression. This goes something like this: we observe closer proximity between modern non-Africans and Neandertals than between modern Africans and Neandertals. This can be explained by either Neandertal-to-non-African gene flow or vice versa.

The authors observe that modern non-Africans are closer to Yoruba than to San. They argue that if Neandertals had modern non-African admixture, then they would also be closer to Yoruba than to San. But, they observe that they are about equidistant to Yoruba and San. Ergo, they couldn't have substantial non-African admixture.

The error in this syllogism is the equation of modern non-Africans (who are closer to Yoruba than to San) with ancient non-Africans for which this was not necessarily the case.

Perhaps a case could be made that this was true for late Out-of-Africans, as these split off from other moderns after the Yoruba-San split. But, the late Out-of-Africans were not the only anatomically modern humans who left Africa, and we have good evidence of anatomically modern humans outside Africa before the exodus that gave rise to modern Eurasians (at sites like Qafzeh).

Note that this observation nicely addresses the argument presented by John Hawks on the young coalescence dates between some Neandertal and Eurasian DNA, which supposedly disqualifies the African structure scenario I presented (and which the authors labeled as "Scenario 4" in their paper). These young coalescence ages can be easily explained as modern-to-Neandertal gene flow within the context of the African structure model.

Finally, I want to remind readers of a very interesting paper I blogged about last year. Here are the relevant quotes:

Our data on neighbors and variability is unsupportive of the strict forms of a single-origin model but does not conflict with another approach, the model of ‘‘isolation by distance,’’ which predicts that genetic and phenotypic dissimilarity increases with geographic distance (24, 29–31). The metapopulation framework would predict the same because frequency and magnitude of genetic exchange would follow the likelihood of 2 populations to meet, which declines with geographical distance from the early AMH epicenter in Africa. Our fossil AMH data, however, suggest that before there was isolation by distance from Africa, there already existed (at least temporally) isolation by distance within Africa during the Pleistocene.

Seemingly ancient contributions to the modern human gene pool (36) have been explained by admixture with archaic forms of Homo, e.g., Neanderthals. Although we cannot rule out such admixture (37), the clear morphological distinction between
AMH and archaic forms of Homo in the light of the proposed ancestral population structure of early AMH to us suggests another underestimated possibility: the genetic exchange between subdivided populations of early AMH as a potential source for ‘‘ancient’’ contributions to the modern human gene pool (9, 36).

I won't present the reasoning behind these conclusions (the paper is open access anyway), but they have an important implication: seemingly "ancient" contributions to modern humans need not have been acquired by either Neandertals or other archaic (pre-sapiens) populations, but they could also have been acquired by admixture with different branches of anatomically modern humans.

Given that anatomically modern humans appear on the record ~200ky ago, so they probably existed at an even earlier date, while extant sapiens populations trace their earlier split much later (the authors date the Yoruba-San split at most 164ky or as late as 67ky), we can reasonably assume that there were anatomically modern sapiens populations that are not ancestral to any modern humans, and which may have contributed seemingly "ancient" genes to our expanding ancestors.

There you have it: structure in Africa, modern-to-Neandertal admixture, or even sapiens-to-sapiens admixture, there are plenty of alternatives to the story dominating the media.