Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

The Doubly Conditioned Frequency Spectrum Does Not Distinguish between Ancient Population Structure and Hybridization

The Doubly Conditioned Frequency Spectrum Does Not Distinguish between Ancient Population... Distinguishing between hybridization and population structure in the ancestral species is a key challenge in our under- standing of how permeable species boundaries are to gene flow. The doubly conditioned frequency spectrum (dcfs) has been argued to be a powerful metric to discriminate between these two explanations, and it was used to argue for hybridization between Neandertal and anatomically modernhumans. Theshape of theobserveddcfsfor thesetwo species cannot be reproduced by a model that represents ancient population structure in Africa with two populations, while adding hybridization produces realistic shapes. In this letter, we show that this result is a consequence of the spatial coarseness of the demographic model and that a spatially structured stepping stone model can generate realistic dcfs without hybridization. This result highlights how inferences on hybridization between recently diverged species can be strongly affected by the choice of how population structure is represented in the underlying demographic model. We also conclude that the dcfs has limited power in distinguishing between the signals left by hybridization and ancient structure. Key words: hybridization, Neandertal, demography, population structure. Hybridization between different species can play a major role either the Eurasian or the African match the Neandertal (but in evolution, both by bringing novel adaptations into species not both) and where the Neandertal is different from the as well as by acting as a barrier to their divergence (Seehausen chimp. D is calculated as the fraction of such sites where 2004; Abbott et al. 2013). However, detecting hybridization the Eurasian genome matches the Neandertal minus the frac- from genetic data can be challenging, as it requires distin- tion where the African genome matches Neandertal. In a guishing actual gene flow after the species split from shared simple four-population model without hybridization, we variation that was present in the ancestral species (Abbott expect Eurasian and African genomes to have the same prob- et al. 2013; Smith and Kronforst 2013; Sousa and Hey 2013). ability of matching the Neandertal through incomplete line- This problem is particularly challenging when considering hy- age sorting, but hybridization between Neandertal and one of bridization among recently diverged species, where past pop- the modern human populations would give rise to an unbal- ulation structure in the ancestral species can leave genetic ance (Green et al. 2010). An analysis using Patterson’s D re- signatures that are almost identical to those left by hy- vealed that the observed values for Neandertal were more bridization (Green et al. 2010; Eriksson and Manica 2012; extreme than expected by chance and were taken as evidence Lowery et al. 2013). for hybridization (Green et al. 2010). This test has been used in The challenges of distinguishing between actual hy- a number of other taxa, such as primates (Pru ¨fer et al. 2012), bridization and ancient population structure have been high- flycatchers (Rheindt et al. 2013), and Heliconius butterflies lighted by the recent publication of Neandertal genomes (Martin et al. 2013). However, a problem in interpreting (Green et al. 2010; Pru ¨fer et al. 2013). The main finding Patterson’s D is that ancestral population structure can pro- coming out of the first analysis of the draft sequence of the duce patterns undistinguishable from hybridization (Durand Neandertal genome (Green et al. 2010) was that populations et al. 2011). In the case of Neandertal, a spatially structured of anatomically modern humans (AMHs) differed in genetic model with realistic demographic parameters can produce D similarity to Neandertal. Specifically, modern Europeans and values identical to the ones measured from real genomes, Asians were significantly more genetically similar to this homi- even in the absence of hybridization (Eriksson and Manica ninthanAfricans(Green et al. 2010). Patterson’s D statistics 2012). (SOM 15 in Green et al. 2010) is arguably the best-known In an attempt to increase the power to detect hy- approach to quantify this pattern. This statistics is based on a bridization, Yang et al. (2012) focused on the frequency dis- panel of four individuals and focuses on biallelic sites where tribution of Neandertal alleles in Eurasian populations at The Author 2014. Published by Oxford University Press on behalf of the Society for Molecular Biology and Evolution. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommon- s.org/licenses/by/3.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work Open Access is properly cited. 1618 Mol. Biol. Evol. 31(6):1618–1621 doi:10.1093/molbev/msu103 Advance Access publication March 13, 2014 Downloaded from https://academic.oup.com/mbe/article/31/6/1618/2925736 by DeepDyve user on 16 July 2022 Ancient Population Structure and Hybridization doi:10.1093/molbev/msu103 MBE Fig. 1. (a) A schematic representation on how the sample frequency of the Neandertal allele of a doubly conditioned locus is calculated. A locus is doubly conditioned if chimp and Neandertal have different alleles (shown in blue and red, respectively), and the ancestral chimp (blue) allele is found in Africa. The frequency of the Neandertal (red) allele is then estimated in the Eurasian panel: in this example, the frequency is 3. (b)Observeddcfs(the dcfs depicts the relative abundance of doubly conditioned loci with different derived allele frequencies), as estimated by Yang et al. (2012). Photographs from Wikipedia Commons, taken by T. Lersch, T. Evanson, W. Warby, Dyor, P. Neo, J. Montrasio, Y. Picq, and Fae. biallelic loci where Neandertal differ from the chimpanzee informative metric to distinguish between hybridization reference genome and modern-day Africans have the and ancient population structure, and this result has been chimp allele. These loci have been called “doubly condi- taken as a confirmation of hybridization between Neandertal tioned,” as they need to have the same allele in a modern and AMHs (e.g., Sankararaman et al. 2012). African genome and the chimp genome (first condition) but However, it remains to be determined whether the dcfs to differ between chimp and Neandertal genomes (second can distinguish between hybridization and ancient structure condition; see fig. 1a for a schematic representation). Such when a spatially structured model with multiple populations loci should, in principle, be enriched for mutations that oc- is used instead of Yang et al.’s representation of ancient struc- curred in the Neandertal line and subsequently entered the ture in the whole Africa continent with only two populations. human line through hybridization, and their relative fre- Such spatially structured models better capture the global quency (the doubly conditioned frequency spectrum, dcfs, genetic clines in within-population genetic diversity observed shown in fig. 1b) should be an informative measure of the in AMHs (Prugnolle et al. 2005; Ramachandran et al. 2005). strength of hybridization. Yang et al. (2012) showed that a Here we use the same spatially structured stepping stone population genetics model that represents ancient structure model as previously presented in Eriksson and Manica in Africa with two populations (see fig. 2a and b for a graph- (2012) to explore the properties of the dcfs with a fine- ical representation of this model) predicts a deficit of rare scale representation of ancient structure (fig. 2c,see supple- doubly conditioned alleles (e.g., of frequency one in the mentary material S1, Supplementary Material online, for de- sample) compared to the frequencies estimated from real tails). Realistic demographic parameters were obtained by data. Adding hybridization to such a model, however, re- fitting the stepping stone to match worldwide patterns of stored the appropriate shape of the doubly conditioned spatial differentiation among modern populations and were allele frequency spectrum. Thus, the dcfs seems to be an further subsetted to focus on parameter combinations that 1619 Downloaded from https://academic.oup.com/mbe/article/31/6/1618/2925736 by DeepDyve user on 16 July 2022 Eriksson and Manica doi:10.1093/molbev/msu103 MBE (a)(b)(c) Fig. 2. (a) Schematic representation of the “two-population model” in tree format. The ancestor of Neandertal and AMHs is structured into two populations. Neandertal splits from one of these two populations. The two populations keep exchanging migrants as they become AMHs, until that exchange decreases (but does not stop) when one population (the descendant of the parent population of Neandertal) leaves Africa to colonize Eurasia. (b) Block representation of the “two-population model,” where each block represents a population. (c) Schematic representation of the spatially structured model used in our analysis. The ancestor of Neandertal and AMHs is represented by a chain of interconnected populations with migration rate m (rather than just two as in the other model). The chain is separated into two when Neandertal speciates 320 kya, without any change in demographic parameters. Eventually, the African range becomes AMH at t , when its demography changes and the migration rate becomes m.At modern t , AMHs expand into Eurasia from the demes that were closest to the Neandertal range (note that the separation between Africa and Eurasia is exit generated by the range expansion and not by a change in migration rates, which stay at m throughout the AMH range). (a)(b) Simulated dcfs Empirical dcfs FIG.3. (a) Doubly conditioned frequency spectrum of Neandertal alleles in five Europeans. Circles represent the empirical dcfs observed in the data by Yang et al (2012), and the colored bars show the distribution predicted by our spatially structured model of ancient population structure. The shaded lines show predictions for ten different parameter combinations among the good fits. For comparison, we show Yang et al.’s best model of ancient population structure (green line) and admixture (blue line). In contrast to simple demographic models, our spatial model correctly captures the relative abundance of rare alleles (frequencies of 1 and 2 in the sample). (b) Schematic representation of how spatial structure occasionally prevents a Eurasian lineage (in red) from coalescing back with other Eurasian and Africa lineages (in blue), generating a rare doubly conditioned locus. The key mutation generating the Neandertal-like allele is highlighted by a red star. Note that time on the Neandertal branch was compressed to make room for the out-of- Africa expansion. predicted D between Africans and Europeans to be within of very common alleles, but there are a large number of 0.0020 U of the observed value 0.0457. This simple spatial combinations that fit the observed dcfs almost perfectly model, which does not include any hybridization, predicts (ten examples are shown as lines in fig. 3a, gray lines; see frequency spectra of doubly conditioned alleles (the dcfs) SOM for details). This spatially explicit model (which has that are in line with observed values (gray lines and shaded eight free parameters) provides a fit that is comparable 2 2 ranges in fig. 3a), matching closely the empirical proportion of (R = 99.2 vs. R = 99.7%) to the admixture model in Yang et rare alleles (giving R = 99.2% for the best fit). Some demo- al (2012) (which has nine free parameters; blue line in fig. 3a). graphic parameter combinations give rise to a slight excess It is also considerably better than the best model fit for European dcfs Downloaded from https://academic.oup.com/mbe/article/31/6/1618/2925736 by DeepDyve user on 16 July 2022 Ancient Population Structure and Hybridization doi:10.1093/molbev/msu103 MBE ancient population structure presented in Yang et al (2012), (grant BB/H005854/1). D. Daversa, C. Jiggins, and two anon- which has an R = 93.7% (green line in fig. 3a). The large pro- ymous referees provided useful comments on the portion of rare doubly conditioned alleles in our spatially manuscript. structured model is a consequence of deep splits in gene genealogies, with old, relatively rare lineages being preserved References by the fine-grained spatial structure in the model (fig. 3b). In Abbott R, Albach D, Ansell S, Arntzen JW, Baird SJ, Bierne N, Boughman other words, the presence of multiple (spatially structured) J, Brelsford A, Buerkle CA, Buggs R, et al. 2013. Hybridization and populations within Africa prevents lineages from coalescing speciation. J Evol Biol. 26:229–246. too quickly, thereby allowing for a few European lineages to Durand EY, Patterson N, Reich D, Slatkin M. 2011. Testing for ancient merge back with Neandertal before meeting any African lin- admixture between closely related populations. MolBiolEvol. 28: eage. In many cases, such lineages are only represented by one 2239–2252. Eriksson A, Manica A. 2012. Effect of ancient population structure on or two individuals, giving an excess of rare doubly conditioned the degree of polymorphism shared between modern human pop- loci. ulations and ancient hominins. Proc Natl Acad Sci U S A. 109: It is beyond the scope of this short letter to provide a 13956–13960. formal test for alternative hybridization scenarios with Green RE,KrauseJ,BriggsAW, MaricicT,Stenzel U, KircherM,Patterson Neandertal. Population structure affects a number of aspects N, Li H, Zhai W, Fritz MH, et al. 2010. A draft sequence of the of the similarities between Eurasians and Neandertal. For ex- Neandertal genome. Science 328:710–722. Lowery RK, Uribe G, Jimenez EB, Weiss MA, Herrera KJ, Regueiro M, ample, the degree of matching between ancient and derived Herrera RJ. 2013. Neanderthal and Denisova genetic affinities with SNPs in candidate regions for hybridization (SOM 17 in Green contemporary humans: introgression versus common ancestral et al. [2010]) can be reproduced by a spatial model analogous polymorphisms. Gene 530:83–94. to the one presented in this letter, without any hybridization Martin SH, Dasmahapatra KK, Nadeau NJ, Salazar C, Walters JR, Simpson (Eriksson and Manica 2012). A number of studies, including F, Blaxter M, Manica A, Mallet J, Jiggins CD. 2013. Genome-wide evidence for speciation with gene flow in Heliconius butterflies. the first analyses of two new Neandertal genomes (Pru ¨fer Genome Res. 23:1817–1828. et al. 2013), provides an intricate picture of possible hy- Pru ¨fer K, Munch K, Hellmann I, Akagi K, Miller JR, Walenz B, Koren S, bridization events among a number of hominins. Possibly, Sutton G, Kodira C, Winer R, et al. 2012. The bonobo genome the clearest analysis pointing to hybridization is the dating compared with the chimpanzee and human genomes. Nature of the Neandertal gene flow into modern humans based on 486:527–531. Pru ¨fer K, Racimo F, Patterson N, Jay F, Sankararaman S, Sawyer S, Heinze linkage disequilibrium patterns (Sankararaman et al. 2012). A, Renaud G, Sudmant PH, de Filippo C, et al. 2013. The complete However, such dates are based on the same demographic genome sequence of a Neanderthal from the Altai Mountains. representation used in Yang et al. (2012). Thus, it will be Nature 505:43–49. interesting to see whether linkage disequilibrium patterns Prugnolle F, Manica A, Balloux F. 2005. Geography predicts neutral ge- are affected by different spatial representations of population netic diversity of human populations. Curr Biol. 15:R159–R160. structure or not. Ramachandran S, Deshpande O, Roseman CC, Rosenberg NA, Feldman MW, Cavalli-Sforza LL. 2005. Support from the relationship of ge- In general, the very different results obtained by a model netic and geographic distance in human populations for a serial that represents genetic structure in Africa with two popula- founder effect originating in Africa. Proc Natl Acad Sci U S A. 102: tions (Yang et al. 2012) versus our spatially structured model 15942–15947. highlight the importance of the coarseness at which space is Rheindt FE, Fujita MK, Wilton PR, Edwards SV. 2013. Introgression and described. When investigating hybridization, especially in the phenotypic assimilation in Zimmerius flycatchers (Tyrannidae): pop- ulation genetic and phylogenetic inferences from genome-wide case of recently diverged species, metrics have been devised to SNPs. Syst Biol. 63:134–152. focus the power of the analysis on the key signals that would Sankararaman S, Patterson N, Li H, Pa ¨a ¨bo S, Reich D. 2012. The date of be expected from hybridization. However, spatial structuring interbreeding between Neandertals and modern humans. PLoS of populations can easily mimic such signals. No matter how Genet. 8:e1002947. sophisticated the metrics are, the properties of different de- Seehausen O. 2004. Hybridization and adaptive radiation. Trends Ecol mographic models should be explored, in particular how Evol. 19:198–207. Smith J, Kronforst MR. 2013. Do Heliconius butterfly species exchange robust the analysis is to the spatial scale of demographic mimicry alleles? Biol Lett. 9:20130503. processes. Sousa V, Hey J. 2013. Understanding the origin of species with genome- scale data: modelling gene flow. Nat. Rev Genet. 14:404–414. Acknowledgments Yang MA, Malaspinas A-S, Durand EY, Slatkin M. 2012. Ancient struc- This work was supported by the Leverhume Trust and the ture in Africa unlikely to explain Neanderthal and non-African genetic similarity. Mol Biol Evol. 29:2987–2995. Biotechnology and Biological Sciences Research Council http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Molecular Biology and Evolution Oxford University Press

The Doubly Conditioned Frequency Spectrum Does Not Distinguish between Ancient Population Structure and Hybridization

Download PDF
4 pages

Loading next page...
 
/lp/oxford-university-press/the-doubly-conditioned-frequency-spectrum-does-not-distinguish-between-syNVQyjwXP

References (33)

Publisher
Oxford University Press
Copyright
Copyright © 2022 Society for Molecular Biology and Evolution
ISSN
0737-4038
eISSN
1537-1719
DOI
10.1093/molbev/msu103
pmid
24627034
Publisher site
See Article on Publisher Site

Abstract

Distinguishing between hybridization and population structure in the ancestral species is a key challenge in our under- standing of how permeable species boundaries are to gene flow. The doubly conditioned frequency spectrum (dcfs) has been argued to be a powerful metric to discriminate between these two explanations, and it was used to argue for hybridization between Neandertal and anatomically modernhumans. Theshape of theobserveddcfsfor thesetwo species cannot be reproduced by a model that represents ancient population structure in Africa with two populations, while adding hybridization produces realistic shapes. In this letter, we show that this result is a consequence of the spatial coarseness of the demographic model and that a spatially structured stepping stone model can generate realistic dcfs without hybridization. This result highlights how inferences on hybridization between recently diverged species can be strongly affected by the choice of how population structure is represented in the underlying demographic model. We also conclude that the dcfs has limited power in distinguishing between the signals left by hybridization and ancient structure. Key words: hybridization, Neandertal, demography, population structure. Hybridization between different species can play a major role either the Eurasian or the African match the Neandertal (but in evolution, both by bringing novel adaptations into species not both) and where the Neandertal is different from the as well as by acting as a barrier to their divergence (Seehausen chimp. D is calculated as the fraction of such sites where 2004; Abbott et al. 2013). However, detecting hybridization the Eurasian genome matches the Neandertal minus the frac- from genetic data can be challenging, as it requires distin- tion where the African genome matches Neandertal. In a guishing actual gene flow after the species split from shared simple four-population model without hybridization, we variation that was present in the ancestral species (Abbott expect Eurasian and African genomes to have the same prob- et al. 2013; Smith and Kronforst 2013; Sousa and Hey 2013). ability of matching the Neandertal through incomplete line- This problem is particularly challenging when considering hy- age sorting, but hybridization between Neandertal and one of bridization among recently diverged species, where past pop- the modern human populations would give rise to an unbal- ulation structure in the ancestral species can leave genetic ance (Green et al. 2010). An analysis using Patterson’s D re- signatures that are almost identical to those left by hy- vealed that the observed values for Neandertal were more bridization (Green et al. 2010; Eriksson and Manica 2012; extreme than expected by chance and were taken as evidence Lowery et al. 2013). for hybridization (Green et al. 2010). This test has been used in The challenges of distinguishing between actual hy- a number of other taxa, such as primates (Pru ¨fer et al. 2012), bridization and ancient population structure have been high- flycatchers (Rheindt et al. 2013), and Heliconius butterflies lighted by the recent publication of Neandertal genomes (Martin et al. 2013). However, a problem in interpreting (Green et al. 2010; Pru ¨fer et al. 2013). The main finding Patterson’s D is that ancestral population structure can pro- coming out of the first analysis of the draft sequence of the duce patterns undistinguishable from hybridization (Durand Neandertal genome (Green et al. 2010) was that populations et al. 2011). In the case of Neandertal, a spatially structured of anatomically modern humans (AMHs) differed in genetic model with realistic demographic parameters can produce D similarity to Neandertal. Specifically, modern Europeans and values identical to the ones measured from real genomes, Asians were significantly more genetically similar to this homi- even in the absence of hybridization (Eriksson and Manica ninthanAfricans(Green et al. 2010). Patterson’s D statistics 2012). (SOM 15 in Green et al. 2010) is arguably the best-known In an attempt to increase the power to detect hy- approach to quantify this pattern. This statistics is based on a bridization, Yang et al. (2012) focused on the frequency dis- panel of four individuals and focuses on biallelic sites where tribution of Neandertal alleles in Eurasian populations at The Author 2014. Published by Oxford University Press on behalf of the Society for Molecular Biology and Evolution. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommon- s.org/licenses/by/3.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work Open Access is properly cited. 1618 Mol. Biol. Evol. 31(6):1618–1621 doi:10.1093/molbev/msu103 Advance Access publication March 13, 2014 Downloaded from https://academic.oup.com/mbe/article/31/6/1618/2925736 by DeepDyve user on 16 July 2022 Ancient Population Structure and Hybridization doi:10.1093/molbev/msu103 MBE Fig. 1. (a) A schematic representation on how the sample frequency of the Neandertal allele of a doubly conditioned locus is calculated. A locus is doubly conditioned if chimp and Neandertal have different alleles (shown in blue and red, respectively), and the ancestral chimp (blue) allele is found in Africa. The frequency of the Neandertal (red) allele is then estimated in the Eurasian panel: in this example, the frequency is 3. (b)Observeddcfs(the dcfs depicts the relative abundance of doubly conditioned loci with different derived allele frequencies), as estimated by Yang et al. (2012). Photographs from Wikipedia Commons, taken by T. Lersch, T. Evanson, W. Warby, Dyor, P. Neo, J. Montrasio, Y. Picq, and Fae. biallelic loci where Neandertal differ from the chimpanzee informative metric to distinguish between hybridization reference genome and modern-day Africans have the and ancient population structure, and this result has been chimp allele. These loci have been called “doubly condi- taken as a confirmation of hybridization between Neandertal tioned,” as they need to have the same allele in a modern and AMHs (e.g., Sankararaman et al. 2012). African genome and the chimp genome (first condition) but However, it remains to be determined whether the dcfs to differ between chimp and Neandertal genomes (second can distinguish between hybridization and ancient structure condition; see fig. 1a for a schematic representation). Such when a spatially structured model with multiple populations loci should, in principle, be enriched for mutations that oc- is used instead of Yang et al.’s representation of ancient struc- curred in the Neandertal line and subsequently entered the ture in the whole Africa continent with only two populations. human line through hybridization, and their relative fre- Such spatially structured models better capture the global quency (the doubly conditioned frequency spectrum, dcfs, genetic clines in within-population genetic diversity observed shown in fig. 1b) should be an informative measure of the in AMHs (Prugnolle et al. 2005; Ramachandran et al. 2005). strength of hybridization. Yang et al. (2012) showed that a Here we use the same spatially structured stepping stone population genetics model that represents ancient structure model as previously presented in Eriksson and Manica in Africa with two populations (see fig. 2a and b for a graph- (2012) to explore the properties of the dcfs with a fine- ical representation of this model) predicts a deficit of rare scale representation of ancient structure (fig. 2c,see supple- doubly conditioned alleles (e.g., of frequency one in the mentary material S1, Supplementary Material online, for de- sample) compared to the frequencies estimated from real tails). Realistic demographic parameters were obtained by data. Adding hybridization to such a model, however, re- fitting the stepping stone to match worldwide patterns of stored the appropriate shape of the doubly conditioned spatial differentiation among modern populations and were allele frequency spectrum. Thus, the dcfs seems to be an further subsetted to focus on parameter combinations that 1619 Downloaded from https://academic.oup.com/mbe/article/31/6/1618/2925736 by DeepDyve user on 16 July 2022 Eriksson and Manica doi:10.1093/molbev/msu103 MBE (a)(b)(c) Fig. 2. (a) Schematic representation of the “two-population model” in tree format. The ancestor of Neandertal and AMHs is structured into two populations. Neandertal splits from one of these two populations. The two populations keep exchanging migrants as they become AMHs, until that exchange decreases (but does not stop) when one population (the descendant of the parent population of Neandertal) leaves Africa to colonize Eurasia. (b) Block representation of the “two-population model,” where each block represents a population. (c) Schematic representation of the spatially structured model used in our analysis. The ancestor of Neandertal and AMHs is represented by a chain of interconnected populations with migration rate m (rather than just two as in the other model). The chain is separated into two when Neandertal speciates 320 kya, without any change in demographic parameters. Eventually, the African range becomes AMH at t , when its demography changes and the migration rate becomes m.At modern t , AMHs expand into Eurasia from the demes that were closest to the Neandertal range (note that the separation between Africa and Eurasia is exit generated by the range expansion and not by a change in migration rates, which stay at m throughout the AMH range). (a)(b) Simulated dcfs Empirical dcfs FIG.3. (a) Doubly conditioned frequency spectrum of Neandertal alleles in five Europeans. Circles represent the empirical dcfs observed in the data by Yang et al (2012), and the colored bars show the distribution predicted by our spatially structured model of ancient population structure. The shaded lines show predictions for ten different parameter combinations among the good fits. For comparison, we show Yang et al.’s best model of ancient population structure (green line) and admixture (blue line). In contrast to simple demographic models, our spatial model correctly captures the relative abundance of rare alleles (frequencies of 1 and 2 in the sample). (b) Schematic representation of how spatial structure occasionally prevents a Eurasian lineage (in red) from coalescing back with other Eurasian and Africa lineages (in blue), generating a rare doubly conditioned locus. The key mutation generating the Neandertal-like allele is highlighted by a red star. Note that time on the Neandertal branch was compressed to make room for the out-of- Africa expansion. predicted D between Africans and Europeans to be within of very common alleles, but there are a large number of 0.0020 U of the observed value 0.0457. This simple spatial combinations that fit the observed dcfs almost perfectly model, which does not include any hybridization, predicts (ten examples are shown as lines in fig. 3a, gray lines; see frequency spectra of doubly conditioned alleles (the dcfs) SOM for details). This spatially explicit model (which has that are in line with observed values (gray lines and shaded eight free parameters) provides a fit that is comparable 2 2 ranges in fig. 3a), matching closely the empirical proportion of (R = 99.2 vs. R = 99.7%) to the admixture model in Yang et rare alleles (giving R = 99.2% for the best fit). Some demo- al (2012) (which has nine free parameters; blue line in fig. 3a). graphic parameter combinations give rise to a slight excess It is also considerably better than the best model fit for European dcfs Downloaded from https://academic.oup.com/mbe/article/31/6/1618/2925736 by DeepDyve user on 16 July 2022 Ancient Population Structure and Hybridization doi:10.1093/molbev/msu103 MBE ancient population structure presented in Yang et al (2012), (grant BB/H005854/1). D. Daversa, C. Jiggins, and two anon- which has an R = 93.7% (green line in fig. 3a). The large pro- ymous referees provided useful comments on the portion of rare doubly conditioned alleles in our spatially manuscript. structured model is a consequence of deep splits in gene genealogies, with old, relatively rare lineages being preserved References by the fine-grained spatial structure in the model (fig. 3b). In Abbott R, Albach D, Ansell S, Arntzen JW, Baird SJ, Bierne N, Boughman other words, the presence of multiple (spatially structured) J, Brelsford A, Buerkle CA, Buggs R, et al. 2013. Hybridization and populations within Africa prevents lineages from coalescing speciation. J Evol Biol. 26:229–246. too quickly, thereby allowing for a few European lineages to Durand EY, Patterson N, Reich D, Slatkin M. 2011. Testing for ancient merge back with Neandertal before meeting any African lin- admixture between closely related populations. MolBiolEvol. 28: eage. In many cases, such lineages are only represented by one 2239–2252. Eriksson A, Manica A. 2012. Effect of ancient population structure on or two individuals, giving an excess of rare doubly conditioned the degree of polymorphism shared between modern human pop- loci. ulations and ancient hominins. Proc Natl Acad Sci U S A. 109: It is beyond the scope of this short letter to provide a 13956–13960. formal test for alternative hybridization scenarios with Green RE,KrauseJ,BriggsAW, MaricicT,Stenzel U, KircherM,Patterson Neandertal. Population structure affects a number of aspects N, Li H, Zhai W, Fritz MH, et al. 2010. A draft sequence of the of the similarities between Eurasians and Neandertal. For ex- Neandertal genome. Science 328:710–722. Lowery RK, Uribe G, Jimenez EB, Weiss MA, Herrera KJ, Regueiro M, ample, the degree of matching between ancient and derived Herrera RJ. 2013. Neanderthal and Denisova genetic affinities with SNPs in candidate regions for hybridization (SOM 17 in Green contemporary humans: introgression versus common ancestral et al. [2010]) can be reproduced by a spatial model analogous polymorphisms. Gene 530:83–94. to the one presented in this letter, without any hybridization Martin SH, Dasmahapatra KK, Nadeau NJ, Salazar C, Walters JR, Simpson (Eriksson and Manica 2012). A number of studies, including F, Blaxter M, Manica A, Mallet J, Jiggins CD. 2013. Genome-wide evidence for speciation with gene flow in Heliconius butterflies. the first analyses of two new Neandertal genomes (Pru ¨fer Genome Res. 23:1817–1828. et al. 2013), provides an intricate picture of possible hy- Pru ¨fer K, Munch K, Hellmann I, Akagi K, Miller JR, Walenz B, Koren S, bridization events among a number of hominins. Possibly, Sutton G, Kodira C, Winer R, et al. 2012. The bonobo genome the clearest analysis pointing to hybridization is the dating compared with the chimpanzee and human genomes. Nature of the Neandertal gene flow into modern humans based on 486:527–531. Pru ¨fer K, Racimo F, Patterson N, Jay F, Sankararaman S, Sawyer S, Heinze linkage disequilibrium patterns (Sankararaman et al. 2012). A, Renaud G, Sudmant PH, de Filippo C, et al. 2013. The complete However, such dates are based on the same demographic genome sequence of a Neanderthal from the Altai Mountains. representation used in Yang et al. (2012). Thus, it will be Nature 505:43–49. interesting to see whether linkage disequilibrium patterns Prugnolle F, Manica A, Balloux F. 2005. Geography predicts neutral ge- are affected by different spatial representations of population netic diversity of human populations. Curr Biol. 15:R159–R160. structure or not. Ramachandran S, Deshpande O, Roseman CC, Rosenberg NA, Feldman MW, Cavalli-Sforza LL. 2005. Support from the relationship of ge- In general, the very different results obtained by a model netic and geographic distance in human populations for a serial that represents genetic structure in Africa with two popula- founder effect originating in Africa. Proc Natl Acad Sci U S A. 102: tions (Yang et al. 2012) versus our spatially structured model 15942–15947. highlight the importance of the coarseness at which space is Rheindt FE, Fujita MK, Wilton PR, Edwards SV. 2013. Introgression and described. When investigating hybridization, especially in the phenotypic assimilation in Zimmerius flycatchers (Tyrannidae): pop- ulation genetic and phylogenetic inferences from genome-wide case of recently diverged species, metrics have been devised to SNPs. Syst Biol. 63:134–152. focus the power of the analysis on the key signals that would Sankararaman S, Patterson N, Li H, Pa ¨a ¨bo S, Reich D. 2012. The date of be expected from hybridization. However, spatial structuring interbreeding between Neandertals and modern humans. PLoS of populations can easily mimic such signals. No matter how Genet. 8:e1002947. sophisticated the metrics are, the properties of different de- Seehausen O. 2004. Hybridization and adaptive radiation. Trends Ecol mographic models should be explored, in particular how Evol. 19:198–207. Smith J, Kronforst MR. 2013. Do Heliconius butterfly species exchange robust the analysis is to the spatial scale of demographic mimicry alleles? Biol Lett. 9:20130503. processes. Sousa V, Hey J. 2013. Understanding the origin of species with genome- scale data: modelling gene flow. Nat. Rev Genet. 14:404–414. Acknowledgments Yang MA, Malaspinas A-S, Durand EY, Slatkin M. 2012. Ancient struc- This work was supported by the Leverhume Trust and the ture in Africa unlikely to explain Neanderthal and non-African genetic similarity. Mol Biol Evol. 29:2987–2995. Biotechnology and Biological Sciences Research Council

Journal

Molecular Biology and Evolution – Oxford University Press

Published: Jun 1, 2014

There are no references for this article.