Applying methods of the global burden of diseases, injuries, and risk factors study to developmental neurotoxicants: a commentary

Applying methods of the global burden of diseases, injuries, and risk factors study to... The purpose of this commentary is to consider whether the methods of the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) can provide accurate estimates of the impact of developmental neurotoxicant exposures on population health. The discussion focuses on two concerns. First, GBD implicitly largely endorses a “high risk” or “disease” approach to estimating health loss rather than a “population-based” approach. Exposure to many developmental neurotoxicants is highly prevalent but, for most individuals, it does not affect functional health to such an extent that diagnostic criteria for a disease are met. Nevertheless, the impacts are real and can be substantial when viewed in terms of the aggregate impact on a population. Second, in GBD the disability weights used for the most common sequelae of developmental neurotoxicant exposures, based on judgments provided by general population respondents, are not commensurate with the import that these sequelae have for an individual’s lifelong well-being, including their ability to fulfill educational, occupational, and social potential. It would be unfortunate if priorities were set or policy decisions made based on how developmental neurotoxicants compare to other risk factors using the current GBD methods. Keywords: Burden of disease, Developmental neurotoxicity, Subclinical toxicity, Disability weighting, Health loss Background The purpose of this commentary is to examine whether The data generated over nearly three decades by the such the GBD approach to characterizing “health loss” Global Burden of Diseases, Injuries, and Risk Factors provides an accurate characterization of the impact of Study (GBD) have been revolutionary in that a common developmental neurotoxicants on population health. metric, the Disability-Adjusted Life Year (DALY), is used to make direct comparisons within a population of the Main text premature mortality and functional health losses attrib- Subclinical neurotoxicity utable to a wide array of diseases and risk factors (e.g., To date, relatively few studies have estimated the burden [1]). GBD studies are thus intended to provide public of disease associated with environmental chemical expo- health authorities with data that can be used to sures. In analyses of lead [2] and methylmercury [3], the prioritize those interventions that will provide the only health loss considered to contribute to the DALY greatest returns, in terms of health gains, on the invest- total, at least with regard to developmental neurotoxi- ment of limited resources. city, was an exposure-related reduction in IQ, specific- For any method that can be applied as broadly as that ally to a value below 70 (two standard deviations below of the GBD, there are likely to be contexts in which the expected population mean). In the ICD framework, some of the key assumptions are not fully appropriate. this is the criterion for identifying an individual with an “intellectual disability.” In the GBD 2015 analysis, in which lead was the only neurodevelopmental toxicant Correspondence: david.bellinger@childrens.harvard.edu Boston Children’s Hospital, 300 Longwood Avenue, Boston, MA 02115, USA considered, “borderline intellectual disability” was also © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Bellinger Environmental Health (2018) 17:53 Page 2 of 6 included, corresponding roughly to IQ scores of 70–84. the relevant question is not whether an individual “has It is certainly true that exposure to a neurotoxicant can it,” but rather, “how much of it.” Fig. 1 also shows that cause the IQ scores of some children to fall below 70, the cumulative frequency distributions of the high and but this would be expected to occur in a relatively small low dentine lead groups never crossed, only meeting percentage of the exposed population, either individuals when they finally converged at 100%. This leftward shift with extremely high exposures that cause frank brain of the entire distribution of IQ scores, i.e., toward lower pathology or individuals who, because of the presence of values, means that the exposure-associated reduction in other significant risk factors, had an IQ score only a few IQ did not occur only among children functioning at the points above 70 prior to the neurotoxicant exposure. lower end of the distribution, but also occurred among Nevertheless, exposure-related changes in the numbers children throughout the distribution. A complete reck- of individuals who fall into the lower portion of a per- oning of the burden attributable to lead must consider formance distribution have frequently been used to illus- not only how the two distributions differ in their ex- trate the impact of a neurotoxicant on a population. For treme tails but also the total area of the differences example, Needleman et al. [4] showed that the fre- between the distributions. The exposure-associated re- quency of a Verbal IQ score below 80 was increased ductions in the IQ scores of the children in the middle three-fold among children with “ahigh” versus “low” of the distribution, where vast majority of children fall, concentration of lead in the dentine layer of shed de- are, in aggregate, likely to contribute more to the total ciduous teeth (Fig. 1). burden attributable to lead exposure than do the A fundamental problem with defining a health loss as exposure-associated reductions in the relatively small occurring only when an individual’s IQ score falls below number of children in the extreme lower tail of the IQ a certain value is that it lacks a logical foundation. There distribution. Bellinger [5] illustrated this by quantifying is no coherent rationale for considering a reduction in the impact, at the population level, of this leftward IQ from 73 to 68 be a health loss but not a reduction shift of the distribution, calculating the total number from 76 to 71. Use of cut-offs is necessary for a number of IQ points lost from lead exposure in the cohort of of purposes, including to prioritize the allocation of U.S. children less than 5 years of age. When data on treatment resources, third-party reimbursement, etc., the dose-effect relationship, derived from a set of but “health loss” is not a binary state. For many diseases, pooled analyses of prospective studies [6], were Fig. 1 Verbal IQ Scores and Dentine Lead Concentration. Cumulative frequency distributions of Verbal IQ scores of children with a “low” (≤ 6 μg/g) or “high” (> 24 μg/g) concentration of lead in the dentine layer of deciduous teeth (from “Lead-associated intellectual deficit”, Needleman HL, Leviton A, Bellinger D, Volume 306, page 367. Copyright © 1983) Massachusetts Medical Society. Reprinted with permission Bellinger Environmental Health (2018) 17:53 Page 3 of 6 combined with NHANES data on the distribution of popu- already low performers suffer disproportionately from a lation blood lead concentrations, the total loss of IQ in this given increase in exposure to lead. Therefore, the cohort was nearly 23 million points. This was substantially dose-effect relationship assumed in estimating the im- greater than the losses estimated for many other pediatric pact of IQ loss should take into account the loss of func- diseases and events, including ADHD (17 million points), tion per unit increase in neurotoxicant biomarker at the traumatic brain injury (5 million points), congenital heart relevant functional level of the individual. disease (100,000 points) and brain tumors (35,000 points). The context-dependence of the severity of neurotox- Because of the ubiquity of lead exposure and the absence icity also has implications for the estimation of disease of a threshold for neurotoxicity, nearly half of the total loss burden globally. The distribution of co-morbid factors was contributed by the large number of children with a that modify neurotoxicity differs by region, so a given blood lead concentration less than 2.1 μg/dL. This is a vari- level of exposure might have more dire consequences ation on the principle, articulated by Rose [7], that, “alarge for individuals in low-resource regions where the preva- number of people at a small risk maygiverisetomore lence of some such factors might be greater than in cases of disease than the small number who are at high high-resource regions. Thus, estimation of disease bur- risk” (p.37). It is the cumulative effect of modest but preva- den might require the use of region-specific assumptions lent impacts that are of greatest concern from the about the magnitudes of the expected impacts. In standpoint of population health. Developing a method for addition, some environmental chemicals disrupt endo- capturing the burden imposed by high prevalence-low crine processes that affect brain development. Therefore, morbidity exposures is crucial if the total morbidity associ- the possibility of sex-specific differences in the severity ated with developmental neurotoxicant exposures is to be of neurotoxicity should be considered in estimating estimated accurately. burden. A calculation of the total IQ loss in the cohort of 0 to 5-year-old children from the late 1970’s suggests that the Disability weights public health interventions implemented since the late A critical step in estimating the import of a nonfatal health 1970’s saved approximately 100 million IQ points insofar state for the burden of disease is the assignment of a dis- as the IQ loss attributable to lead in this earlier cohort ability weight (DW). A DW can vary from 0 to 1, where 0 of 0 to 5 year olds was approximately 125 million points. represents optimal health and 1 is equivalent to death. The Given that the number of children in this age group is DW and the duration of the imperfect health state, in com- approximately 25 million, this represents an average in- bination, determine the Years-Lived-with-Disability com- crease of about 5 IQ points per child over this period, ponent of the DALY total. i.e., one-third of a standard deviation. Independent ana- In the initial GBD studies, DWs were derived using lyses suggest that the mean IQ score of U.S. adults has economic evaluation methods. Health professionals increased by 4 to 5 points over this same period. provided judgments about the severity distribution of Justification for considering a reduction in IQ as a health states and the social preference for time lived at health loss no matter where it occurs in the distribution different severity levels. In effect, these were judgments is provided by the methods used in econometric ana- about the import of the health state for quality of life, lyses, in which each IQ point lost results in a reduction social desirability, and the value of the life of an af- in lifetime earnings. For instance, in an analysis con- fected individual. Responding to critiques of this ap- ducted using EU data, each IQ point lost was assumed proach, from GBD 2010 onwards the judgments about to reduce lifetime earnings by 17,363 Euros [8]. the value of departures from ideal health have been Burden of disease analyses for neurotoxic chemicals provided by individuals from the general population in- must incorporate several considerations that might be stead of health professionals. The method currently less relevant to other risk factors. Although in econo- used involves the presentation of paired descriptions, in metric analyses the productivity loss per IQ point is as- lay terms using 30 words or fewer, of individuals in two sumed to be the same across the entire IQ distribution health states. The respondent is asked to decide, “who (i.e., the reduction an individual’s productivity is the is healthier.” same whether IQ drops from 120 to 119 or 75 to 74), The DWs assigned to certain health states linked to this might be incorrect. In terms of import for future developmental neurotoxicant exposures appear to be well-being, the loss of one IQ point for an individual greatly affected the training or experience of respon- whose pre-exposure IQ was low because of the presence dents. Table 1 lists the DWs for intellectual disability of other risk factors might result in a greater reduction (ID) of different levels of severity used in GBD studies of future productivity than would the loss of one point between 2004 and 2015 [9]. Health professionals, the re- for an individual whose pre-exposure IQ was spondents in GBD 2004, clearly regarded intellectual dis- higher. Also, there is evidence that individuals who are ability of all severity levels to entail a greater health loss Bellinger Environmental Health (2018) 17:53 Page 4 of 6 Table 1 Changes over time in disability weights for severity Table 2 Lay descriptions for different levels of severity of levels of intellectual disability in GBD studies intellectual disability: GBD 2015 Health state GBD 2015 GBD 2010 GBD 2004 Severity levels Lay descriptions Borderline intellectual functioning 0.011 0.0034 Borderline intellectual Is slow in learning at school. As an adult, functioning the person has some difficulty doing Intellectual disability-mild 0.043 0.031 0.290 complex or unfamiliar tasks but otherwise functions independently. Intellectual disability-moderate 0.100 0.080 0.430 ID/mental retardation: mild Has low intelligence and is slow in Intellectual disability-severe 0.160 0.126 0.820 learning at school. As an adult, the Intellectual disability-profound 0.200 0.157 0.760 person can live independently, but often needs help to raise children and can only work at simple supervised jobs than did members of the general population respondents ID/mental retardation: Has low intelligence, and is slow in in GBD 2010 and GBD 2015. moderate learning to speak and to do even One possible explanation for the discrepancies is that simple tasks. As an adult, the person requires a lot of support to live health professionals and members of the general popula- independently and raise children. tion used different definitions of “health” in making their The person can only work at the judgments. Perhaps members of the general population simplest supervised jobs. consider an individual with ID to be disabled, but in ID/mental retardation: Has very low intelligence and cannot otherwise excellent health because he or she does not severe speak more than a few words, needs constant supervision and help with have a “disease” as this term is understood by laymen. In most daily activities, and can do only contrast, health professionals might have interpreted the the simplest tasks. construct of health more broadly, taking into account ID/mental retardation: Has very low intelligence, has almost not only physical health at a specific point in time, but profound no language, and does not understand also current and future well-being, including an individ- even the most basic requests or instructions. The person requires constant ual’s ability to carry out activities of daily living and to supervision and help for all activities fulfill his or her educational, occupational, and social po- tential. Such a broad interpretation of health is consistent with the definition of health articulated in the WHO con- individual is unable to function independently in any stitution, “…a state of complete physical, mental and social capacity, has a DW lower than that of concussion well-being and not merely the absence of disease or in- (0.214) and fracture of the pelvis-short term (0.279). firmity.” From this perspective, the DWs assigned to the As noted earlier, the duration of a disease contributes different levels of ID in the most recent GBD studies are to the YLD, so a health state with a DW similar to that surprisingly small, despite the fact that the lay descriptions of one of the categories of ID but which is easily treated, clearly enumerate the limitations of individuals at each ID with full recovery and only rare complications (e.g., severity level (Table 2). epididymo-orchitis), will produce many fewer DALYs The descriptions do not specify the IQ ranges associ- than an essentially untreatable, life-long condition such ated with each category, but in the ICD10, mild ID cor- as ID. On the other hand, if a thumb were amputated in responds to an IQ between 50 and 69; moderate ID to childhood, its YLD would be similar to the YLD of bor- an IQ between 35 and 49; severe ID to an IQ between derline intellectual functioning, despite the fact that the 20 and 34; and profound ID to an IQ below 20. two individuals would face distinctly different challenges The view that the DWs for ID in GBD 2015 underesti- over their lifespans. mate the diverse challenges of affected individuals, is re- In order to fully capture the burden imposed on an in- inforced by considering other health states with similar dividual by the sequelae of exposure to a developmental DWs. The DW of borderline intellectual functioning neurotoxicant, the disease model applied must include (0.011) is comparable those of amputation of a thumb not only the individual’s physical health at a given (0.011), ear pain (0.013), abdominopelvic problem-mild point-in-time, but also the extent to which exposure (0.011), dental caries: symptomatic (0.010), and generic limits the likelihood that the individual will be able to uncomplicated disease: anxiety about diagnosis (0.012). reach his or her educational, occupational, and social po- Health states with a DW similar to ID/mental retarda- tential in subsequent years. The need to consider such a tion:moderate (0.100) include diabetic neuropathy developmental perspective, extending over the life (0.133), epididymo-orchitis (0.128), neck pain-moderate course, is one way in which the health effects of a devel- (0.114), infectious disease: acute episode-severe (0.133), opmental neurotoxicant differ from those of many other and amputation of one arm: long-term with or without risk factors considered in GBD studies. Although a re- treatment (0.118). Even ID/mental retardation:profound duction in IQ is the endpoint most often measured in (DW of 0.200), with a lay description indicating that an developmental neurotoxicity studies, and has been Bellinger Environmental Health (2018) 17:53 Page 5 of 6 considered in GBD studies, it is really only the “tip of the impact of ADHD and only accommodations are the iceberg,” serving as a proxy for the myriad other available to mitigate the impact of borderline intellec- neurodevelopmental impairments that are sequelae. tual functioning. One challenge in applying GBD Moreover, rather than being the end of the story for an methods to developmental neurotoxicants is that be- affected child, a reduction in IQ might be only the be- cause the constellations of sequela might not be recog- ginning, and a failure to consider how life unfolds over nized as disease states, DWs are not available for them. time for the child with an impaired IQ will result in a One such example is chronic metallic mercury vapor gross underestimate of the magnitude of the impact that intoxication. Steckling and colleagues used expert elicit- developmental neurotoxicant exposure has had, given ation and a systematic review to develop a description that early-life exposures to neurotoxicants might initiate of this health state and generated a DW by asking ex- developmental cascades that, over time, manifest in pert respondents to complete a pairwise comparison other adverse health states. Although the lay descrip- exercise. The resulting DW was then used to estimate tions of the different severity levels of ID in GBD 2015 the global disease burden associated with mercury in- refer to the limitations that these states place on an toxication caused by artisanal gold mining [11]. adult’s functioning, the respondents providing the judg- In a GBD study, only risk factor-outcome associations ments on which DWs are based did not appear to apply for which the evidence regarding causality is considered a life-course perspective. A recent longitudinal study in convincing or probable should be included. Currently, New Zealand provides a good example of the import- the World Cancer Research Fund grades of evidence are ance of such a perspective. In that study, a child’s blood used to identify associations that meet this criterion [1]. lead concentration at age 11 years was inversely related In this framework, greatest weight is given to evidence not only to IQ in adulthood, at 38 years of age, but also from randomized controlled trials, non-randomized to socioeconomic status (SES) [10]. Greater lead expos- intervention studies, and prospective observational ure in childhood was associated with reduced upward studies. In studying developmental neurotoxicants, or social mobility, and the SES of an individual with higher a environmental health risks in general, it is generally not blood lead concentration in childhood tended to be lower feasible to conduct either randomized trials or interven- than that of his or her parents. The link reported between tional studies of risk-outcome associations. Therefore, early lead exposure and anti-social behavior in adulthood the absence of evidence from such studies should not be provides another example of a developmental cascade, ini- weighted heavily in assessing the likelihood that such as- tiated by reduced IQ and mediated by a variety of down- sociations reflect causality. Other systematic review stream sequelae, including impairment of impulse control, strategies that include consideration of evidence from inability to delay gratification, lack of educational achieve- non-human and toxicologic studies, as well as from hu- ment, ADHD, and substance abuse. man studies, would be more appropriate [12]. While disability weights are available for health states other than ID that have been associated with develop- Conclusion mental exposures to neurotoxicants (Table 3), to date Current GBD methods will not fully characterize the im- these health states have not been included in GBD studies pacts of developmental neurotoxicants on population of such exposures. It is surprising that the DW for ADHD health. It would be unfortunate if priorities were set or is about 4 times that of borderline intellectual functioning, policy decisions made based on how developmental neu- even though effective treatments are available to reduce rotoxicants compare to other risk factors on the basis of DALYs calculated using these methods. While the results generated by applying these methods would be valid, Table 3 Disability Weights for Health States Relevant to they would capture only a small fraction of the total Developmental Neurotoxicants health impact of such exposures. In effect, the GBD Health state GBD 2015 GBD 2010 GBD 2004 method endorses “a high-risk” approach to disease pre- ADHD 0.045 0.049 0.20 vention rather than a “population-based” approach. In Conduct disorder 0.241 0.236 0.150 an approach from the latter perspective, subclinical im- Asperger’s syndrome 0.104 0.110 pacts would be considered to represent a health loss be- cause of the toll they exact on an individual’s current Autism 0.262 0.259 0.550 and future well-being. Because exposure to some devel- Hearing loss-mild 0.010 0.005 0.040 opmental neurotoxicants is ubiquitous, very large num- Speech problems 0.051 0.054 bers of individuals suffer such impacts, resulting in a Motor impairment-mild 0.010 0.012 0.010 large cumulative toll on the well-being of a population. Motor plus cognitive 0.031 0.054 0.024 In addition, a burden analysis that is based solely on re- impairment-mild duction in IQ scores ignores the contributions of the Bellinger Environmental Health (2018) 17:53 Page 6 of 6 other health states that might occur in the future as a re- Received: 13 April 2018 Accepted: 24 May 2018 sult of a cascade of adversities initiated by childhood ex- posure to a neurotoxicant. A second concern about the References application of GBD methods to developmental neuro- 1. GBD 2016 Risk Factor Collaborators. Global, regional, and national comparative risk assessment of 84 behavioural, environmental and toxicants is the underestimation of the magnitude of the occupational, and metabolic risks or clusters of risks, 1990–2016: a disability associated with health states that are the most systematic analysis for the Global Burden of Disease Study 2016. Lancet. common sequelae of these exposures. The explanation is 2017;390:1345–442. 2. Fewtrell LJ, Prüss-Ustün A, Landrigan P, Ayuso-Mateos JL. Estimating the not certain, but it might reflect a greater weighting of global burden of disease of mild mental retardation and cardiovascular physical illnesses than limitations that prevent an indi- diseases from environmental lead exposure. Environ Res. 2004;94(2):120–33. vidual from achieving as much success as an adult as he 3. Poulin J, Gibb H. Mercury assessing the environmental burden of disease at national and local levels, Environmental Burden of Disease Series No. 16. or she might otherwise have achieved. An approach Geneva: World Health Organization; 2008. based on a human capital framework, which both en- 4. Needleman HL, Leviton A, Bellinger D. Lead-associated intellectual deficit. N compasses, “the knowledge, skills, competencies, and at- Engl J Med. 1982;306:367. 5. Bellinger DC. A strategy for comparing the contributions of environmental tributes embodied in individuals that facilitate the chemicals and other risk factors to neurodevelopment of children. Environ creation of personal, social and economic well-being,” Health Perspect. 2012;120:501–7. ([13], p.18), and acknowledges the “dynamic comple- 6. Lanphear BP, Hornung R, Khoury J, Yolton K, Baghurst P, Bellinger DC, Canfield RL, Dietrich KN, Bornschein R, Greene T, Rothenberg SJ, Needleman HL, mentarities” implicit in the concept of developmental Schnaas L, Wasserman G, Graziano J, Roberts R. Low-level environmental lead cascades holds greater promise of fully estimating the exposure and children's intellectual function: an international pooled analysis. disease burden attributable to neurotoxicants. It is not Environ Health Perspect. 2005;113(7):894–9. 7. Rose G. Sick individuals and sick populations. Inter J Epidemiol. 1985;14:32–8. clear that current GBD methods can accommodate such 8. Bellanger M, Pichery C, Aerts D, Berglund M, Castano A, Cejchanova M, an approach. One of the important advantages of a GBD Crettaz P, Davidson F, Esteban M, Fischer ME, Gurzau AE, Halzlova K, analysis is that it permits direct comparison of the health Katsonouri A, Knudsen LE, Kolossa-Gehring M, Koppen G, Ligocka D, Miklavcic A, Reis MF, Rudnai P, Tratnik JS, Weihe P, Budtz-Jorgensen E, burdens associated with diverse risk factors or diseases. Grandjean P. Economic benefits of methylmercury control in Europe: Developing methods that remedy the problems of GBD monetary value of neurotoxicity prevention. Environ Health. 2013;12:3. methods as they pertain to developmental neurotoxi- 9. World Health Organization. WHO methods and data sources for global burden of disease estimates 2000–2015, Global Health Estimates Technical cants without applying them to other risk factors and Paper WHO/HIS/IER/GHE/2017.1. Department of Information, Evidence, and diseases would eliminate this advantage, so careful Research. Geneva: World Health Organization; 2017. thought needs to be given to how to retain the ability to 10. Reuben A, Caspi A, Belsky DW, Broadbent J, Harrington H, Sugden K, Houts RM, Ramrakha S, Poulton R, Moffitt TE. Association of childhood blood lead conduct comparative assessments but, at the same time, levels with cognitive function and socioeconomic status at age 38 years provide estimates of the disease burden that are more and with IQ change and socioeconomic mobility between childhood and policy-relevant. adulthood. JAMA. 2017;317(12):1244–51. 11. Steckling N, Tobollik M, Plass D, Hornberg C, Ericson B, Fuller R, Bose-O'Reilly S. Global burden of disease of mercury used in artisanal small-scale gold mining. Abbreviations Ann Glob Health. 2017;83(2):234–47. ADHD: Attention Deficit Hyperactivity Disorder; DALY: Disability-Adjusted Life- 12. Lam J, Koustas E, Sutton P, Johnson PI, Atchley DS, Sen S, Robinson KA, Year; DW: Disability weight; GBD: Global Burden of Disease; ICD: International Axelrad DA, Woodruff TJ. The navigation guide - evidence-based medicine Classification of Disease; ID: Intellectual disability; IQ: Intelligence Quotient; meets environmental health: integration of animal and human evidence for NHANES: National Health and Nutrition Examination Survey; OECD: Organisation PFOA effects on fetal growth. Environ Health Perspect. 2014;122:1040–51. of Economic Co-operation and Development; WHO: World Health Organization; 13. Organisation for Economic Co-operation and Development. The well-being YLD: Years lived with disability of nations: the role of human and social capital. Paris, 2001. http://www. oecd.org/site/worldforum/33703702.pdf. Acknowledgments Julia Matthews, PhD, MD provided a critical review of the manuscript. Availability of data and materials Data sharing not applicable to this article as no datasets were generated or analysed during the current study. Authors’ contributions DCB drafted the manuscript. The author read and approved the final manuscript. Ethics approval and consent to participate Not applicable Competing interests The author declares that he has no competing interests. Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Environmental Health Springer Journals

Applying methods of the global burden of diseases, injuries, and risk factors study to developmental neurotoxicants: a commentary

Free
6 pages
Loading next page...
 
/lp/springer_journal/applying-methods-of-the-global-burden-of-diseases-injuries-and-risk-LkLmyd6atN
Publisher
BioMed Central
Copyright
Copyright © 2018 by The Author(s).
Subject
Environment; Environmental Health; Occupational Medicine/Industrial Medicine; Public Health
eISSN
1476-069X
D.O.I.
10.1186/s12940-018-0397-7
Publisher site
See Article on Publisher Site

Abstract

The purpose of this commentary is to consider whether the methods of the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) can provide accurate estimates of the impact of developmental neurotoxicant exposures on population health. The discussion focuses on two concerns. First, GBD implicitly largely endorses a “high risk” or “disease” approach to estimating health loss rather than a “population-based” approach. Exposure to many developmental neurotoxicants is highly prevalent but, for most individuals, it does not affect functional health to such an extent that diagnostic criteria for a disease are met. Nevertheless, the impacts are real and can be substantial when viewed in terms of the aggregate impact on a population. Second, in GBD the disability weights used for the most common sequelae of developmental neurotoxicant exposures, based on judgments provided by general population respondents, are not commensurate with the import that these sequelae have for an individual’s lifelong well-being, including their ability to fulfill educational, occupational, and social potential. It would be unfortunate if priorities were set or policy decisions made based on how developmental neurotoxicants compare to other risk factors using the current GBD methods. Keywords: Burden of disease, Developmental neurotoxicity, Subclinical toxicity, Disability weighting, Health loss Background The purpose of this commentary is to examine whether The data generated over nearly three decades by the such the GBD approach to characterizing “health loss” Global Burden of Diseases, Injuries, and Risk Factors provides an accurate characterization of the impact of Study (GBD) have been revolutionary in that a common developmental neurotoxicants on population health. metric, the Disability-Adjusted Life Year (DALY), is used to make direct comparisons within a population of the Main text premature mortality and functional health losses attrib- Subclinical neurotoxicity utable to a wide array of diseases and risk factors (e.g., To date, relatively few studies have estimated the burden [1]). GBD studies are thus intended to provide public of disease associated with environmental chemical expo- health authorities with data that can be used to sures. In analyses of lead [2] and methylmercury [3], the prioritize those interventions that will provide the only health loss considered to contribute to the DALY greatest returns, in terms of health gains, on the invest- total, at least with regard to developmental neurotoxi- ment of limited resources. city, was an exposure-related reduction in IQ, specific- For any method that can be applied as broadly as that ally to a value below 70 (two standard deviations below of the GBD, there are likely to be contexts in which the expected population mean). In the ICD framework, some of the key assumptions are not fully appropriate. this is the criterion for identifying an individual with an “intellectual disability.” In the GBD 2015 analysis, in which lead was the only neurodevelopmental toxicant Correspondence: david.bellinger@childrens.harvard.edu Boston Children’s Hospital, 300 Longwood Avenue, Boston, MA 02115, USA considered, “borderline intellectual disability” was also © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Bellinger Environmental Health (2018) 17:53 Page 2 of 6 included, corresponding roughly to IQ scores of 70–84. the relevant question is not whether an individual “has It is certainly true that exposure to a neurotoxicant can it,” but rather, “how much of it.” Fig. 1 also shows that cause the IQ scores of some children to fall below 70, the cumulative frequency distributions of the high and but this would be expected to occur in a relatively small low dentine lead groups never crossed, only meeting percentage of the exposed population, either individuals when they finally converged at 100%. This leftward shift with extremely high exposures that cause frank brain of the entire distribution of IQ scores, i.e., toward lower pathology or individuals who, because of the presence of values, means that the exposure-associated reduction in other significant risk factors, had an IQ score only a few IQ did not occur only among children functioning at the points above 70 prior to the neurotoxicant exposure. lower end of the distribution, but also occurred among Nevertheless, exposure-related changes in the numbers children throughout the distribution. A complete reck- of individuals who fall into the lower portion of a per- oning of the burden attributable to lead must consider formance distribution have frequently been used to illus- not only how the two distributions differ in their ex- trate the impact of a neurotoxicant on a population. For treme tails but also the total area of the differences example, Needleman et al. [4] showed that the fre- between the distributions. The exposure-associated re- quency of a Verbal IQ score below 80 was increased ductions in the IQ scores of the children in the middle three-fold among children with “ahigh” versus “low” of the distribution, where vast majority of children fall, concentration of lead in the dentine layer of shed de- are, in aggregate, likely to contribute more to the total ciduous teeth (Fig. 1). burden attributable to lead exposure than do the A fundamental problem with defining a health loss as exposure-associated reductions in the relatively small occurring only when an individual’s IQ score falls below number of children in the extreme lower tail of the IQ a certain value is that it lacks a logical foundation. There distribution. Bellinger [5] illustrated this by quantifying is no coherent rationale for considering a reduction in the impact, at the population level, of this leftward IQ from 73 to 68 be a health loss but not a reduction shift of the distribution, calculating the total number from 76 to 71. Use of cut-offs is necessary for a number of IQ points lost from lead exposure in the cohort of of purposes, including to prioritize the allocation of U.S. children less than 5 years of age. When data on treatment resources, third-party reimbursement, etc., the dose-effect relationship, derived from a set of but “health loss” is not a binary state. For many diseases, pooled analyses of prospective studies [6], were Fig. 1 Verbal IQ Scores and Dentine Lead Concentration. Cumulative frequency distributions of Verbal IQ scores of children with a “low” (≤ 6 μg/g) or “high” (> 24 μg/g) concentration of lead in the dentine layer of deciduous teeth (from “Lead-associated intellectual deficit”, Needleman HL, Leviton A, Bellinger D, Volume 306, page 367. Copyright © 1983) Massachusetts Medical Society. Reprinted with permission Bellinger Environmental Health (2018) 17:53 Page 3 of 6 combined with NHANES data on the distribution of popu- already low performers suffer disproportionately from a lation blood lead concentrations, the total loss of IQ in this given increase in exposure to lead. Therefore, the cohort was nearly 23 million points. This was substantially dose-effect relationship assumed in estimating the im- greater than the losses estimated for many other pediatric pact of IQ loss should take into account the loss of func- diseases and events, including ADHD (17 million points), tion per unit increase in neurotoxicant biomarker at the traumatic brain injury (5 million points), congenital heart relevant functional level of the individual. disease (100,000 points) and brain tumors (35,000 points). The context-dependence of the severity of neurotox- Because of the ubiquity of lead exposure and the absence icity also has implications for the estimation of disease of a threshold for neurotoxicity, nearly half of the total loss burden globally. The distribution of co-morbid factors was contributed by the large number of children with a that modify neurotoxicity differs by region, so a given blood lead concentration less than 2.1 μg/dL. This is a vari- level of exposure might have more dire consequences ation on the principle, articulated by Rose [7], that, “alarge for individuals in low-resource regions where the preva- number of people at a small risk maygiverisetomore lence of some such factors might be greater than in cases of disease than the small number who are at high high-resource regions. Thus, estimation of disease bur- risk” (p.37). It is the cumulative effect of modest but preva- den might require the use of region-specific assumptions lent impacts that are of greatest concern from the about the magnitudes of the expected impacts. In standpoint of population health. Developing a method for addition, some environmental chemicals disrupt endo- capturing the burden imposed by high prevalence-low crine processes that affect brain development. Therefore, morbidity exposures is crucial if the total morbidity associ- the possibility of sex-specific differences in the severity ated with developmental neurotoxicant exposures is to be of neurotoxicity should be considered in estimating estimated accurately. burden. A calculation of the total IQ loss in the cohort of 0 to 5-year-old children from the late 1970’s suggests that the Disability weights public health interventions implemented since the late A critical step in estimating the import of a nonfatal health 1970’s saved approximately 100 million IQ points insofar state for the burden of disease is the assignment of a dis- as the IQ loss attributable to lead in this earlier cohort ability weight (DW). A DW can vary from 0 to 1, where 0 of 0 to 5 year olds was approximately 125 million points. represents optimal health and 1 is equivalent to death. The Given that the number of children in this age group is DW and the duration of the imperfect health state, in com- approximately 25 million, this represents an average in- bination, determine the Years-Lived-with-Disability com- crease of about 5 IQ points per child over this period, ponent of the DALY total. i.e., one-third of a standard deviation. Independent ana- In the initial GBD studies, DWs were derived using lyses suggest that the mean IQ score of U.S. adults has economic evaluation methods. Health professionals increased by 4 to 5 points over this same period. provided judgments about the severity distribution of Justification for considering a reduction in IQ as a health states and the social preference for time lived at health loss no matter where it occurs in the distribution different severity levels. In effect, these were judgments is provided by the methods used in econometric ana- about the import of the health state for quality of life, lyses, in which each IQ point lost results in a reduction social desirability, and the value of the life of an af- in lifetime earnings. For instance, in an analysis con- fected individual. Responding to critiques of this ap- ducted using EU data, each IQ point lost was assumed proach, from GBD 2010 onwards the judgments about to reduce lifetime earnings by 17,363 Euros [8]. the value of departures from ideal health have been Burden of disease analyses for neurotoxic chemicals provided by individuals from the general population in- must incorporate several considerations that might be stead of health professionals. The method currently less relevant to other risk factors. Although in econo- used involves the presentation of paired descriptions, in metric analyses the productivity loss per IQ point is as- lay terms using 30 words or fewer, of individuals in two sumed to be the same across the entire IQ distribution health states. The respondent is asked to decide, “who (i.e., the reduction an individual’s productivity is the is healthier.” same whether IQ drops from 120 to 119 or 75 to 74), The DWs assigned to certain health states linked to this might be incorrect. In terms of import for future developmental neurotoxicant exposures appear to be well-being, the loss of one IQ point for an individual greatly affected the training or experience of respon- whose pre-exposure IQ was low because of the presence dents. Table 1 lists the DWs for intellectual disability of other risk factors might result in a greater reduction (ID) of different levels of severity used in GBD studies of future productivity than would the loss of one point between 2004 and 2015 [9]. Health professionals, the re- for an individual whose pre-exposure IQ was spondents in GBD 2004, clearly regarded intellectual dis- higher. Also, there is evidence that individuals who are ability of all severity levels to entail a greater health loss Bellinger Environmental Health (2018) 17:53 Page 4 of 6 Table 1 Changes over time in disability weights for severity Table 2 Lay descriptions for different levels of severity of levels of intellectual disability in GBD studies intellectual disability: GBD 2015 Health state GBD 2015 GBD 2010 GBD 2004 Severity levels Lay descriptions Borderline intellectual functioning 0.011 0.0034 Borderline intellectual Is slow in learning at school. As an adult, functioning the person has some difficulty doing Intellectual disability-mild 0.043 0.031 0.290 complex or unfamiliar tasks but otherwise functions independently. Intellectual disability-moderate 0.100 0.080 0.430 ID/mental retardation: mild Has low intelligence and is slow in Intellectual disability-severe 0.160 0.126 0.820 learning at school. As an adult, the Intellectual disability-profound 0.200 0.157 0.760 person can live independently, but often needs help to raise children and can only work at simple supervised jobs than did members of the general population respondents ID/mental retardation: Has low intelligence, and is slow in in GBD 2010 and GBD 2015. moderate learning to speak and to do even One possible explanation for the discrepancies is that simple tasks. As an adult, the person requires a lot of support to live health professionals and members of the general popula- independently and raise children. tion used different definitions of “health” in making their The person can only work at the judgments. Perhaps members of the general population simplest supervised jobs. consider an individual with ID to be disabled, but in ID/mental retardation: Has very low intelligence and cannot otherwise excellent health because he or she does not severe speak more than a few words, needs constant supervision and help with have a “disease” as this term is understood by laymen. In most daily activities, and can do only contrast, health professionals might have interpreted the the simplest tasks. construct of health more broadly, taking into account ID/mental retardation: Has very low intelligence, has almost not only physical health at a specific point in time, but profound no language, and does not understand also current and future well-being, including an individ- even the most basic requests or instructions. The person requires constant ual’s ability to carry out activities of daily living and to supervision and help for all activities fulfill his or her educational, occupational, and social po- tential. Such a broad interpretation of health is consistent with the definition of health articulated in the WHO con- individual is unable to function independently in any stitution, “…a state of complete physical, mental and social capacity, has a DW lower than that of concussion well-being and not merely the absence of disease or in- (0.214) and fracture of the pelvis-short term (0.279). firmity.” From this perspective, the DWs assigned to the As noted earlier, the duration of a disease contributes different levels of ID in the most recent GBD studies are to the YLD, so a health state with a DW similar to that surprisingly small, despite the fact that the lay descriptions of one of the categories of ID but which is easily treated, clearly enumerate the limitations of individuals at each ID with full recovery and only rare complications (e.g., severity level (Table 2). epididymo-orchitis), will produce many fewer DALYs The descriptions do not specify the IQ ranges associ- than an essentially untreatable, life-long condition such ated with each category, but in the ICD10, mild ID cor- as ID. On the other hand, if a thumb were amputated in responds to an IQ between 50 and 69; moderate ID to childhood, its YLD would be similar to the YLD of bor- an IQ between 35 and 49; severe ID to an IQ between derline intellectual functioning, despite the fact that the 20 and 34; and profound ID to an IQ below 20. two individuals would face distinctly different challenges The view that the DWs for ID in GBD 2015 underesti- over their lifespans. mate the diverse challenges of affected individuals, is re- In order to fully capture the burden imposed on an in- inforced by considering other health states with similar dividual by the sequelae of exposure to a developmental DWs. The DW of borderline intellectual functioning neurotoxicant, the disease model applied must include (0.011) is comparable those of amputation of a thumb not only the individual’s physical health at a given (0.011), ear pain (0.013), abdominopelvic problem-mild point-in-time, but also the extent to which exposure (0.011), dental caries: symptomatic (0.010), and generic limits the likelihood that the individual will be able to uncomplicated disease: anxiety about diagnosis (0.012). reach his or her educational, occupational, and social po- Health states with a DW similar to ID/mental retarda- tential in subsequent years. The need to consider such a tion:moderate (0.100) include diabetic neuropathy developmental perspective, extending over the life (0.133), epididymo-orchitis (0.128), neck pain-moderate course, is one way in which the health effects of a devel- (0.114), infectious disease: acute episode-severe (0.133), opmental neurotoxicant differ from those of many other and amputation of one arm: long-term with or without risk factors considered in GBD studies. Although a re- treatment (0.118). Even ID/mental retardation:profound duction in IQ is the endpoint most often measured in (DW of 0.200), with a lay description indicating that an developmental neurotoxicity studies, and has been Bellinger Environmental Health (2018) 17:53 Page 5 of 6 considered in GBD studies, it is really only the “tip of the impact of ADHD and only accommodations are the iceberg,” serving as a proxy for the myriad other available to mitigate the impact of borderline intellec- neurodevelopmental impairments that are sequelae. tual functioning. One challenge in applying GBD Moreover, rather than being the end of the story for an methods to developmental neurotoxicants is that be- affected child, a reduction in IQ might be only the be- cause the constellations of sequela might not be recog- ginning, and a failure to consider how life unfolds over nized as disease states, DWs are not available for them. time for the child with an impaired IQ will result in a One such example is chronic metallic mercury vapor gross underestimate of the magnitude of the impact that intoxication. Steckling and colleagues used expert elicit- developmental neurotoxicant exposure has had, given ation and a systematic review to develop a description that early-life exposures to neurotoxicants might initiate of this health state and generated a DW by asking ex- developmental cascades that, over time, manifest in pert respondents to complete a pairwise comparison other adverse health states. Although the lay descrip- exercise. The resulting DW was then used to estimate tions of the different severity levels of ID in GBD 2015 the global disease burden associated with mercury in- refer to the limitations that these states place on an toxication caused by artisanal gold mining [11]. adult’s functioning, the respondents providing the judg- In a GBD study, only risk factor-outcome associations ments on which DWs are based did not appear to apply for which the evidence regarding causality is considered a life-course perspective. A recent longitudinal study in convincing or probable should be included. Currently, New Zealand provides a good example of the import- the World Cancer Research Fund grades of evidence are ance of such a perspective. In that study, a child’s blood used to identify associations that meet this criterion [1]. lead concentration at age 11 years was inversely related In this framework, greatest weight is given to evidence not only to IQ in adulthood, at 38 years of age, but also from randomized controlled trials, non-randomized to socioeconomic status (SES) [10]. Greater lead expos- intervention studies, and prospective observational ure in childhood was associated with reduced upward studies. In studying developmental neurotoxicants, or social mobility, and the SES of an individual with higher a environmental health risks in general, it is generally not blood lead concentration in childhood tended to be lower feasible to conduct either randomized trials or interven- than that of his or her parents. The link reported between tional studies of risk-outcome associations. Therefore, early lead exposure and anti-social behavior in adulthood the absence of evidence from such studies should not be provides another example of a developmental cascade, ini- weighted heavily in assessing the likelihood that such as- tiated by reduced IQ and mediated by a variety of down- sociations reflect causality. Other systematic review stream sequelae, including impairment of impulse control, strategies that include consideration of evidence from inability to delay gratification, lack of educational achieve- non-human and toxicologic studies, as well as from hu- ment, ADHD, and substance abuse. man studies, would be more appropriate [12]. While disability weights are available for health states other than ID that have been associated with develop- Conclusion mental exposures to neurotoxicants (Table 3), to date Current GBD methods will not fully characterize the im- these health states have not been included in GBD studies pacts of developmental neurotoxicants on population of such exposures. It is surprising that the DW for ADHD health. It would be unfortunate if priorities were set or is about 4 times that of borderline intellectual functioning, policy decisions made based on how developmental neu- even though effective treatments are available to reduce rotoxicants compare to other risk factors on the basis of DALYs calculated using these methods. While the results generated by applying these methods would be valid, Table 3 Disability Weights for Health States Relevant to they would capture only a small fraction of the total Developmental Neurotoxicants health impact of such exposures. In effect, the GBD Health state GBD 2015 GBD 2010 GBD 2004 method endorses “a high-risk” approach to disease pre- ADHD 0.045 0.049 0.20 vention rather than a “population-based” approach. In Conduct disorder 0.241 0.236 0.150 an approach from the latter perspective, subclinical im- Asperger’s syndrome 0.104 0.110 pacts would be considered to represent a health loss be- cause of the toll they exact on an individual’s current Autism 0.262 0.259 0.550 and future well-being. Because exposure to some devel- Hearing loss-mild 0.010 0.005 0.040 opmental neurotoxicants is ubiquitous, very large num- Speech problems 0.051 0.054 bers of individuals suffer such impacts, resulting in a Motor impairment-mild 0.010 0.012 0.010 large cumulative toll on the well-being of a population. Motor plus cognitive 0.031 0.054 0.024 In addition, a burden analysis that is based solely on re- impairment-mild duction in IQ scores ignores the contributions of the Bellinger Environmental Health (2018) 17:53 Page 6 of 6 other health states that might occur in the future as a re- Received: 13 April 2018 Accepted: 24 May 2018 sult of a cascade of adversities initiated by childhood ex- posure to a neurotoxicant. A second concern about the References application of GBD methods to developmental neuro- 1. GBD 2016 Risk Factor Collaborators. Global, regional, and national comparative risk assessment of 84 behavioural, environmental and toxicants is the underestimation of the magnitude of the occupational, and metabolic risks or clusters of risks, 1990–2016: a disability associated with health states that are the most systematic analysis for the Global Burden of Disease Study 2016. Lancet. common sequelae of these exposures. The explanation is 2017;390:1345–442. 2. Fewtrell LJ, Prüss-Ustün A, Landrigan P, Ayuso-Mateos JL. Estimating the not certain, but it might reflect a greater weighting of global burden of disease of mild mental retardation and cardiovascular physical illnesses than limitations that prevent an indi- diseases from environmental lead exposure. Environ Res. 2004;94(2):120–33. vidual from achieving as much success as an adult as he 3. Poulin J, Gibb H. Mercury assessing the environmental burden of disease at national and local levels, Environmental Burden of Disease Series No. 16. or she might otherwise have achieved. An approach Geneva: World Health Organization; 2008. based on a human capital framework, which both en- 4. Needleman HL, Leviton A, Bellinger D. Lead-associated intellectual deficit. N compasses, “the knowledge, skills, competencies, and at- Engl J Med. 1982;306:367. 5. Bellinger DC. A strategy for comparing the contributions of environmental tributes embodied in individuals that facilitate the chemicals and other risk factors to neurodevelopment of children. Environ creation of personal, social and economic well-being,” Health Perspect. 2012;120:501–7. ([13], p.18), and acknowledges the “dynamic comple- 6. Lanphear BP, Hornung R, Khoury J, Yolton K, Baghurst P, Bellinger DC, Canfield RL, Dietrich KN, Bornschein R, Greene T, Rothenberg SJ, Needleman HL, mentarities” implicit in the concept of developmental Schnaas L, Wasserman G, Graziano J, Roberts R. Low-level environmental lead cascades holds greater promise of fully estimating the exposure and children's intellectual function: an international pooled analysis. disease burden attributable to neurotoxicants. It is not Environ Health Perspect. 2005;113(7):894–9. 7. Rose G. Sick individuals and sick populations. Inter J Epidemiol. 1985;14:32–8. clear that current GBD methods can accommodate such 8. Bellanger M, Pichery C, Aerts D, Berglund M, Castano A, Cejchanova M, an approach. One of the important advantages of a GBD Crettaz P, Davidson F, Esteban M, Fischer ME, Gurzau AE, Halzlova K, analysis is that it permits direct comparison of the health Katsonouri A, Knudsen LE, Kolossa-Gehring M, Koppen G, Ligocka D, Miklavcic A, Reis MF, Rudnai P, Tratnik JS, Weihe P, Budtz-Jorgensen E, burdens associated with diverse risk factors or diseases. Grandjean P. Economic benefits of methylmercury control in Europe: Developing methods that remedy the problems of GBD monetary value of neurotoxicity prevention. Environ Health. 2013;12:3. methods as they pertain to developmental neurotoxi- 9. World Health Organization. WHO methods and data sources for global burden of disease estimates 2000–2015, Global Health Estimates Technical cants without applying them to other risk factors and Paper WHO/HIS/IER/GHE/2017.1. Department of Information, Evidence, and diseases would eliminate this advantage, so careful Research. Geneva: World Health Organization; 2017. thought needs to be given to how to retain the ability to 10. Reuben A, Caspi A, Belsky DW, Broadbent J, Harrington H, Sugden K, Houts RM, Ramrakha S, Poulton R, Moffitt TE. Association of childhood blood lead conduct comparative assessments but, at the same time, levels with cognitive function and socioeconomic status at age 38 years provide estimates of the disease burden that are more and with IQ change and socioeconomic mobility between childhood and policy-relevant. adulthood. JAMA. 2017;317(12):1244–51. 11. Steckling N, Tobollik M, Plass D, Hornberg C, Ericson B, Fuller R, Bose-O'Reilly S. Global burden of disease of mercury used in artisanal small-scale gold mining. Abbreviations Ann Glob Health. 2017;83(2):234–47. ADHD: Attention Deficit Hyperactivity Disorder; DALY: Disability-Adjusted Life- 12. Lam J, Koustas E, Sutton P, Johnson PI, Atchley DS, Sen S, Robinson KA, Year; DW: Disability weight; GBD: Global Burden of Disease; ICD: International Axelrad DA, Woodruff TJ. The navigation guide - evidence-based medicine Classification of Disease; ID: Intellectual disability; IQ: Intelligence Quotient; meets environmental health: integration of animal and human evidence for NHANES: National Health and Nutrition Examination Survey; OECD: Organisation PFOA effects on fetal growth. Environ Health Perspect. 2014;122:1040–51. of Economic Co-operation and Development; WHO: World Health Organization; 13. Organisation for Economic Co-operation and Development. The well-being YLD: Years lived with disability of nations: the role of human and social capital. Paris, 2001. http://www. oecd.org/site/worldforum/33703702.pdf. Acknowledgments Julia Matthews, PhD, MD provided a critical review of the manuscript. Availability of data and materials Data sharing not applicable to this article as no datasets were generated or analysed during the current study. Authors’ contributions DCB drafted the manuscript. The author read and approved the final manuscript. Ethics approval and consent to participate Not applicable Competing interests The author declares that he has no competing interests. Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Journal

Environmental HealthSpringer Journals

Published: Jun 4, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off