Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations

Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities... EI-03128; No of Pages 9 Environment International xxx (2015) xxx–xxx Contents lists available at ScienceDirect Environment International journal homepage: www.elsevier.com/locate/envint Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations a a, b d c e Paul Whaley , Crispin Halsall , Marlene Ågerstrand ,Elisa Aiassa , Diane Benford , Gary Bilotta , f w n g h David Coggon , Chris Collins , Ciara Dempsey , Raquel Duarte-Davidson , Rex FitzGerald , x i j k l m Malyka Galay-Burgos , David Gee , Sebastian Hoffmann , Juleen Lam , Toby Lasserson , Len Levy , n o i p q Steven Lipworth , Sarah Mackenzie Ross , Olwenn Martin , Catherine Meads , Monika Meyer-Baron , r s t u v n James Miller , Camilla Pease , Andrew Rooney ,Alison Sapiets , Gavin Stewart , David Taylor Lancaster Environment Centre, Lancaster University, Lancaster LA1 4YQ, UK Department of Environmental Science and Analytical Chemistry, Stockholm University, SE-106 91, Stockholm, Sweden Food Standards Agency, Aviation House, 125 Kingsway, London WC2B 6NH, UK Assessment and Methodological Support Unit, European Food Safety Authority, Via Carlo Magno 1/a 43126, Parma, Italy Aquatic Research Centre, University of Brighton, Lewes Road, Brighton BN2 4GJ, UK MRC Lifecourse Epidemiology Unit, University of Southampton, MRC Lifecourse Epidemiology Unit, Southampton General Hospital, Southampton SO16 6YD, UK Centre for Radiation, Chemicals and Environmental Hazards, Public Health England, Harwell Science and Innovation Campus, Didcot, Oxfordshire OX11 0RQ, UK Swiss Centre for Applied Human Toxicology, University of Basel, Missionsstrasse 64, 4055 Basel, Switzerland Institute for the Environment, Health and Societies, Brunel University London, Kingston Lane, Uxbridge UB8 3PH, UK Evidence-Based Toxicology Collaboration (EBTC), Stembergring 15, 33106 Paderborn, Germany University of California San Francisco, Program on Reproductive Health and the Environment, San Francisco, CA, USA Cochrane Editorial Unit, Cochrane Central Executive, St Albans House, 57-9 Haymarket, London SW1Y 4QX, UK Institute of Environment, Health, Risks and Futures, School of Energy, Environment and Agrifood, Cranfield University, Cranfield, Bedfordshire MK43 0AL, UK Royal Society of Chemistry, Burlington House, Piccadilly, London W1J 0BA, UK Research Department of Clinical, Educational and Health Psychology, University College London, Gower Street, London WC1E 6BT, UK Health Economics Research Group, Brunel University London, Kingston Lane, Uxbridge UB8 3PH, UK Leibniz Research Centre for Working Environment and Human Factors (IfADo), Neurobehavioural Toxicology, Ardeystr 67, D-44139 Dortmund, Germany Centre for Ecology and Hydrology, Wallingford, Oxfordshire 0X10 8BB, UK Ramboll Environ, 1 Broad Gate, The Headrow, Leeds LS1 8EQ, UK National Institute of Environmental Sciences (NIEHS), National Institutes of Health (NIH), Department of Health and Human Services (DHHS), Research Triangle Park, NC, USA Syngenta Ltd., Jealott's Hill International Research Centre, Bracknell RG42 6EY, UK Centre for Rural Economy, School of Agriculture, Food and Rural Development, University of Newcastle upon Tyne, UK Department of Geography and Environmental Science, School of Archaeology, Geography and Environmental Science, University of Reading, Reading, RG6 6DW, United Kingdom European Centre for Ecotoxicology and Toxicology of Chemicals (ECETOC), Avenue Edmond Van Nieuwenhuyse 2 Bte 8B-1160 Brussels, Belgium article i nfo abstract Article history: Systematic review (SR) is a rigorous, protocol-driven approach designed to minimise error and bias when Received 8 August 2015 summarising the body of research evidence relevant to a specificscientific question. Taking as a comparator the Accepted 2 November 2015 use of SR in synthesising research in healthcare, we argue that SR methods could also pave the way for a “step Available online xxxx change” in the transparency, objectivity and communication of chemical risk assessments (CRA) in Europe and else- where. We suggest that current controversies around the safety of certain chemicals are partly due to limitations in Keywords: current CRA procedures which have contributed to ambiguity about the health risks posed by these substances. We Risk assessment present an overview of how SR methods can be applied to the assessment of risks from chemicals, and indicate how Research synthesis challenges in adapting SR methods from healthcare research to the CRA context might be overcome. Regarding the Environment Chemicals latter, we report the outcomes from a workshop exploring how to increase uptake of SR methods, attended by ex- Systematic review perts representing a wide range of fields related to chemical toxicology, risk analysis and SR. Priorities which were Toxicology identified include: the conduct of CRA-focused prototype SRs; the development of a recognised standard of reporting and conduct for SRs in toxicology and CRA; and establishing a network to facilitate research, communica- tion and training in SR methods. We see this paper as a milestone in the creation of a research climate that fosters communication between experts in CRA and SR and facilitates wider uptake of SR methods into CRA. © 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). ⁎ Corresponding author. E-mail address: [email protected] (C. Halsall). http://dx.doi.org/10.1016/j.envint.2015.11.002 0160-4120/© 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 2 P. Whaley et al. / Environment International xxx (2015) xxx–xxx 1. Introduction considered is usually much wider than in the assessment of healthcare interventions. Thus, when the various types of toxicological research Systematic review (SR) is a rigorous, protocol-driven approach to are combined into a single overall conclusion about the health risks minimising error and bias in the aggregation and appraisal of evidence posed by a chemical exposure, reviewers are challenged with integrat- relevant to answering a research question. SR techniques were initially ing the results from a broad and heterogeneous evidence base. developed in the fields of psychology, social science and health care and In spite of these differences, there is reason for thinking that SR have, since the 1980s, provided a valuable tool for evidence-informed methods can be applied successfully to CRA. For example, techniques decision-making across many domains (Lau et al., 2013). In medicine, for aggregating the results of different study types are already addressed SRs have provided a valuable response to the need for consistent, trans- in various frameworks currently in use in toxicology. These include: In- parent and scientifically-robust interpretations of the results of increas- ternational Agency of Research on Cancer (IARC) Monographs ing numbers of often conflicting studies of the efficacy of healthcare (International Agency for Research on Cancer, 2006); the Navigation interventions. SRs have taken on an increasingly fundamental role Guide (Woodruff and Sutton, 2014); and the US Office for Health As- both in supporting decision-making in healthcare and, by channelling sessment and Translation (OHAT) (Rooney et al., 2014; US National Tox- resources towards questions for which the answers are not yet icology Panel, 2015) – though it should be noted that none of these known, reducing waste in research (Chalmers and Glasziou, 2009; approaches have yet applied SR methods to the exposure assessment Salman et al., 2014). It is now accepted practice in healthcare to use component of CRA. Heterogeneous sources of evidence are a familiar SR methods to assess evidence not only for the efficacy of interventions, challenge in all domains including clinical medicine (Lau et al., 1998), but also on diagnostic tests, prognostics and adverse outcomes. and SR of observational studies has a crucial role in identifying compli- The extension of SR techniques to other fields is based on a mutual cations and side-effects of healthcare interventions (Sterne et al., 2014; need across disciplines to make the best use of existing evidence Higgins and Green, 2011). The need for SR of pre-clinical animal trials of when making decisions, a move for which momentum has been grow- healthcare interventions, in order to better anticipate benefits and ing for several decades. For example, the What Works Clearinghouse harms to humans, is another area in which methods being developed was established in 2002 to apply SR techniques in support of and implemented by a number of groups including SYRCLE American educational policy (US Institute of Education Sciences, (Hooijmans et al., 2012; van Luijk et al., 2014) and CAMARADES 2015), and in 2000 the international Campbell Collaboration research (Macleod et al., 2005; Sena et al., 2014). (Stewart and Schmid, 2015) network was convened to undertake and disseminate systematic re- argue that research synthesis methods (including systematic review) views on the effects of social interventions in diverse fields such as are generic and applicable to any domain if appropriately crime and justice, education, international development and social wel- contextualised. fare (Campbell Collaboration, 2015). Meta-analysis and SR in ecology Given the sometimes controversial outcomes of CRAs and the grow- have contributed to evidence-based environmental policy since the ing public and media profile of the risks that chemicals may pose to mid-1990s (Stewart, 2010); more recently, the Collaboration for Envi- humans and the environment, SR is increasingly viewed as a potentially ronmental Evidence (CEE) has been established to encourage conduct powerful technique in assessing and communicating how likely it is that of SRs on a wide range of environmental topics (Collaboration for a chemical will cause harm. SR methods add transparency, rigour and Environmental Evidence, 2015). objectivity to the process of collecting the most relevant scientificevi- The potential advantages of adapting SR methodology to the field of dence with which to inform policy discussions and could provide a crit- chemical risk assessment (CRA) have also been recognised, with multi- ical tool for organising and appraising the evidence on which chemical ple research groups and organisations either developing and adopting policy decisions are based. (Woodruff and Sutton, 2014; Birnbaum et al., 2013; European Food Consequently, in November 2014 a group of 35 scientists and re- Safety Authority, 2010; Rooney et al., 2014; Aiassa et al., 2015)or searchers from the fields of medicine, toxicology, epidemiology, envi- recommending (US National Research Council, 2014a, 2014b; US Envi- ronmental chemistry, ecology, risk assessment, risk management and ronmental Protection Agency, 2013; Silbergeld and Scherer, 2013; Hoff- SR participated in a one-day workshop to consider the application of mann and Hartung, 2006; Zoeller et al., 2015) the use of SR methods for SR in CRA. The purpose was three-fold: evaluating the association between health effects and chemical expo- 1. Identify from expert practitioners in risk assessment and SR the ob- sures to inform decision-making. There are, however, a number of stacles, in terms of practical challenges and knowledge gaps, to recognised challenges in extending SR methods to CRA, many of implementing SR methods in CRA; which derive from key differences in the evidence base between the healthcare and toxicological sciences. 2. Develop a “roadmap” for overcoming those obstacles and expediting SRs in medicine often focus on direct evidence for benefits and ad- the implementation of SR methods, where appropriate, by the vari- verse effects of healthcare interventions derived from randomised con- ous stakeholders involved in CRA; trolled trials (RCTs) in humans. The evidence base for CRA is generally 3. Establish the foundations of a network to co-ordinate research and more complex, with a need to extrapolate from investigations in ani- activities relating to the implementation of SR methods in CRA. The mals, in vitro and in silico, and then to synthesise findings with those aim would be to support best practise in the application of SR tech- from human studies if available. Furthermore, the human data tend to niques and promote the wider adoption of SR in CRA, both in come from observational studies with greater and more varied potential Europe and elsewhere. for bias and confounding than RCTs, and the range of outcomes to be Participants heard seven presentations about recent developments in SR methods, their application to the risk assessment process, and It is worth drawing a distinction between three sources of bias in the review process. their potential value to policy-makers. There were two break-out ses- There is potential for bias in the conduct of a review (e.g. because of inappropriate sions in which participants were divided into three facilitated groups, methods for identifying and selecting evidence for inclusion in the review); bias because firstly to discuss challenges to implementing SR methods in CRA, and the material available for the review is not representative of the evidence base as a whole then to suggest ways in which the obstacles could be overcome. These (due to selective publication); and bias arising from flaws in the design, conduct, analysis and reporting of individual studies included in the review that can cause the effect of an ideas were discussed in plenary before being summarised, circulated intervention or exposure to be systematically under- or over-estimated. One of the major for comment, and then published in this paper. The Workshop was con- functions of SRs is to minimise bias in the conduct of a review and, as far as possible, to en- ducted under the “Chatham House Rule” such that participants were sure that potential bias from selective publication and methodological flaws in the evi- free to refer to the information presented and discussed, provided dence are properly taken into account when drawing conclusions in response to a research question. they did not attribute it to identifiable individuals or organisations. Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 P. Whaley et al. / Environment International xxx (2015) xxx–xxx 3 The purpose of this overview paper is to present the rationale for ex- challenging to distinguish which opinion is likely to represent the ploring the application of SR methods to CRA, the various experts' views most valid synthesis of the totality of available evidence. on the challenges to implementing SR methods in CRA, and their sug- A recent illustrative example (see Box 1) of when expert scientists gestions for overcoming them. The remaining goals of the meeting are and reputable organisations have come to apparently contradictory ongoing work, including the development of the roadmap concept for conclusions about the likelihood of a chemical causing harm is the publication and the establishment of a network for supporting the use case of bisphenol-A (BPA). BPA is a monomer used in the manufacture of SR in CRA. of the resinous linings of tin cans and other food contact materials such as polycarbonate drinks bottles. It has been banned from use in infant-feed bottles across the EU (European Commission, 1/28/2011) 2. The appeal of SR methods in CRA because of “uncertainties concerning the effect of the exposure of in- fants to Bisphenol A” (European Commission, 5/31/2011b). Chemical risk assessment is a multi-step process leading to a quanti- The European Food Safety Authority (EFSA) considers that current tative characterisation of risk, which can then be used to inform the levels of exposure to BPA present a low risk of harm to the public management of chemical substances so as to ensure that any risks to (European Food Safety Authority, 2015a). The French food regulator human health or the environment are managed optimally. CRAs entail ANSES takes a seemingly different stance on the risks to health posed four fundamental steps: hazard identification; hazard characterisation by BPA (French Agency for Food, Environmental and Occupational (often a dose–response assessment); exposure assessment; and risk Health, and Safety, 4/7/2014), determining there to be a “potential risk characterisation (see Fig. 1). These steps draw on various fields of scien- to the unborn children of exposed pregnant women”. On this basis, tific research including environmental chemistry, toxicology ANSES has proposed classifying BPA as toxic to reproduction in humans (encompassing in vivo, in vitro, ecotoxicological and in silico methods), (French Agency for Food, Environmental and Occupational Health, and ecotoxicology, human epidemiology, and mathematical modelling. Safety, 2013), a proposal which has contributed to the French authori- There are many ways in which errors can occur in the interpretation ties' decision to implement an outright ban on BPA in all food packaging of evidence from these varied disciplines, including failure to consider materials (France, 12/24/2012). While the ban has been challenged by all relevant data, failure to allow appropriately for the strengths and lim- some stakeholders as being disproportionate under EU law itations of individual studies, and over- or underestimating the rele- (Tošenovský, 2014, 2015; Plastics Europe, 2015 ), the Danish National vance of experimental models to real-world scenarios (to name a Food Institute has argued that EFSA has overestimated the safe daily ex- few). Whether the appraisal of evidence is based on objective processes, posure to BPA and that some populations are exposed to BPA at levels or on subjective expert judgement and opinion, may also be an impor- higher than can be considered safe (National Food Institute, Denmark, tant factor in accurate interpretation of evidence: the assessment pro- 2015); a view reflected in the conclusions of some researchers, e.g. cess always requires input from technical experts, which inevitably (Vandenberg et al., 2014) but not others, e.g. (US Food and Drug brings an element of subjectivity to the interpretation of the scientific Administration, 2014). evidence. Different experts may have varying degrees of practical and The example of BPA illustrates the challenges in reaching consensus cognitive access to relevant information, place differing weight on indi- even when interpreting the same evidence base regarding the potential vidual studies and/or strands of evidence that they review and, when toxicity of chemical exposures, either in terms of what is known and working in committee, may be more or less influenced by dominant what is uncertain about the risks to health posed by BPA, and/or what personalities. This can result in misleading conclusions in which the po- response is appropriate to managing those risks and uncertainties. It tential for health risks is overlooked, underestimated or overstated. Fur- also shows how, in the absence of that consensus, there is a danger thermore, if the factors determining their assessment of evidence are that policy on BPA may become disconnected from the evidence base, undocumented, when expert opinions are in conflict it can be very either risking harm to health through continued exposure or incurring Fig. 1. An overview to the chemical risk assessment (CRA) process, whereby risk is a function of hazard and exposure. While SR methods could in principle be applied to all steps of the CRA process, it is the view of the workshop participants that up to this point in time most attention has been focused on the hazard identification and hazard characterisation steps. There are issues around conducting a systematic review for exposure assessment which were not discussed at the workshop, such as the requirement for a very different tool for assessing risk of bias in exposure studies which may necessitate specialised knowledge of analytical/environmental chemistry. Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 4 P. Whaley et al. / Environment International xxx (2015) xxx–xxx Box 1. Examples of conflicting opinions from scientists and government agencies about the risks to health posed by bisphenol-A at current exposure levels. unnecessary economic costs through restricting the use of a chemical 6. Reproducibility, in that the conclusions of the SR process when ap- which is in fact sufficiently safe. It also suggests that if the reasons for plied to the same question and data should ideally produce the disagreement about health risks posed by a chemical are not accessible same answer even when undertaken by different individuals (also to various stakeholders in the debate, it then becomes much more diffi- described as “consistency”). In practise, different experts may reach cult for regulators to credibly resolve controversies about chemical difference conclusions because they will not all make the same safety, potentially undermining their authority in the long term. value judgments about the scope, quality and interpretation of evi- This example highlights the potential for differences in the interpre- dence. Therefore, the process should be sufficiently rigorous that it tation of evidence when assessing chemical toxicity and the need for a is highly likely that scientific judgement would result in the same process that is not only scientifically robust but also transparent, so conclusion independent of the experts involved, and as a minimum that the reasons for any disagreement can be readily identified – includ- the SR process should render transparent the reasons for all ing giving stakeholders greater opportunity to understand when differ- conclusions. ences in policy stem from divergent assessments of risk, and when they stem from divergent opinions as to how those risks are best managed. It It may be perceived that the value of SR methods lies in their provi- also suggests the importance of the following characteristics in risk as- sion of unequivocal assessments of whether or not a chemical will in- sessments that are used to inform risk management decisions: duce specific harm to humans and/or wildlife in given circumstances. In practise, however, this will happen only if the evidence base is suffi- 1. Transparency, in that the basis for the conclusions of the risk assess- ciently extensive, there is unanimity in identification of the problem and ment should be clear (otherwise they may not be trusted and errors in assessment of the quality of the evidence base, and also how the ev- may go undetected). idence is to be interpreted in answering the review question (without 2. Validity, in that CRAs should be sufficiently (though not necessarily this, SRs will also produce different results). Often, the consensus and/ maximally) scientifically robust in their methodology and accurate or information may be relatively limited; in such circumstances, a SR in their estimation of risks and characterisation of attendant uncer- will instead clearly state the limitations of the available data and conse- tainties as to optimise the decisions that must be made in risk quent uncertainties. The value here is in the provision of a comprehen- management. sive and transparent assessment of what is not known and insight into 3. Confidence, providing the user with a clear statement as to the overall the drivers of divergent opinion. From a research perspective, this yields strength of evidence for the conclusions reached and a characterisa- valuable information about how research limitations and knowledge tion of the utility of the evidence for decision-making (e.g. “appropri- gaps contribute to ongoing uncertainty about environmental and health ate for hazard identification but inappropriate for identification of a risks, allowing the subsequent efforts of researchers to be more clearly reference dose”). focused. From a policy perspective, SRs offer a transparent explanation 4. Utility, in that the output of the risk assessment should be in a form as to why there are differences in opinion which can then be communi- that is convenient and intelligible to those who will use it (outputs cated to stakeholders. that are too detailed and complex to validate and readily compre- Overall, SR contributes to achieving consensus not by eliminating hend lead to inefficiency and possibly erroneous decisions). expert judgement, nor by eliminating conflicting opinions about 5. Efficiency, providing a clear justification of the choice of research whether a compound should be banned (for example), but by providing question in the context of efficiently solving a CRA problem. Re- a robust, systematic and transparent framework for reviewing evidence sources for CRA are often limited and it is wasteful to expend unnec- of risks, such that when there is disagreement, the reasons for it are essary effort on aspects of an assessment that will not be critical to clearly visible and the relative merits of differing opinions can be appraised. In this way, it may help to resolve controversies in the decision-making (although for the purposes of transparency and va- lidity, the reasons for focusing on a particular outcome or otherwise interpretation of the science which informs the risk management restricting the evaluation should be explained). process. Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 P. Whaley et al. / Environment International xxx (2015) xxx–xxx 5 3. SR and its application to CRA included studies will be appraised, and the analytical techniques used for combining the results of the included studies. The purposes of the 3.1. Traditional vs. SR methods protocol are to discourage ad-hoc changes to methodology during the review process which may introduce bias, to allow any justifiable meth- SR methods are often contrasted with “traditional”, non-systematic odological changes to be tracked, and also to allow peer-review of the narrative approaches to describing what is and is not already known work that it is proposed, to help ensure the utility and validity of its ob- in relation to a research question. In reality, the distinction between sys- jectives and methods. tematic and narrative review is a crude one, with narrative reviews The final SR itself consists of a statement of the objective, the search encompassing a number of different approaches to reviewing evidence, method, the criteria for including relevant studies for analysis, and the from the caricature of one researcher writing about “my field, from my results of the appraisal of internal validity of the included studies, e.g. standpoint […] using only my data and my ideas, and citing only my implemented as a “risk of bias” assessment in Cochrane Reviews of publications” (Caveman, 2000), to thorough narrative critiques of com- randomised trials (Higgins et al., 2011). The evidence is then synthe- prehensively identified evidence relevant to answering an explicitly ar- sised using statistical meta-analytical techniques, narrative methods ticulated question, as conducted by organisations such as IARC or both (depending on the extent to which meta-analysis is possible) (International Agency for Research on Cancer, 2006). into an overall answer to the research question. An assessment is then Nonetheless, it is worth noting that only relatively recently has it made of the strength of the evidence supporting the answer; in been recognised that traditional narrative reviews are, to varying de- Cochrane reviews, this typically follows the GRADE methodology grees, vulnerable to a range of methodological shortcomings which (Atkins et al., 2004), taking into account overall features of the evidence are likely to bias their summarisation of the evidence base (Chalmers base including risk of bias across the included studies, publication bias et al., 2002). These include selective rather than comprehensive re- in the evidence base, external validity or applicability of the evidence trieval of evidence relevant to the review topic, inconsistent interpreta- to the population of interest, heterogeneity of the evidence, and the tion of the impact of methodological shortcomings on the validity of overall precision of the evidence. This is finally followed by a concluding included studies, and even an absence of clear review objectives or con- interpretation of what the SR as a whole determines is and is not known clusions which are drawn directly from the strengths and limitations of in relation to its objective. the evidence base (Mulrow, 1987; Mignini and Khan, 2006). In this, we emphasise the distinction between a SR and a meta- The presence of these shortcomings seriously challenges the reader's analysis. A meta-analysis pools the results of a number of separate stud- ability to determine the credibility of a review. When there exist multi- ies in a single statistical analysis and may be a component of a SR; how- ple competing reviews, each using opaque methods, it becomes almost ever, it does not necessarily incorporate the full set of methodological impossible to judge their relative merits and therefore to base decisions features which define the SR process (e.g. a meta-analysis may or may on current best available evidence. The consequence is a proliferation of not include an assessment of the internal validity of included studies). conflicting opinions about best practice that fail to take proper account While we acknowledge that some researchers use the terms “system- of the body of research evidence. In the healthcare sciences, this was ini- atic review” and “meta-analysis” interchangeably, we believe the two tially shown by Antman and colleagues when they found that, in approaches should be disambiguated. It is also worth noting that comparison to recommendations of clinical experts, systematic aggre- many reviews employ a combination of narrative and systematic gation of data from existing clinical trials of streptokinase to treat myo- methods; there were differing opinions among workshop participants cardial infarction would have demonstrated benefit some years before as to the extent to which it is reasonable to expect all reviews to fully in- recommendations for its use became commonplace (Antman et al., corporate SR methods. 1992). More recently, cumulative meta-analyses have been shown to be more accurate in summarising current understanding of the size of 3.2. The current status of SR in environmental health, toxicology and CRA effect of a wide range of healthcare interventions than researchers plan- ning new clinical trials who have not used these methods (Clarke et al., While the use of SR methodologies is well established in healthcare 2014). to determine the effect of interventions on health outcomes or the accu- A SR is an approach to reviewing evidence which specifically sets out racy of a diagnostic test, application of SR is relatively novel in the fields to avoid these problems, by methodically attempting “to collate all em- of toxicology and environmental health. Workshop participants heard pirical evidence that fits pre-specified eligibility criteria in order to an- how methods for SR of medical interventions have in the United swer a specific research question,” using “explicit, systematic methods States been adapted in both academic and federal contexts to the gath- that are selected with a view to minimising bias” (Higgins and Green, ering and appraising of evidence for the effects of chemical exposures 2011). on human health: researchers at the University of California have devel- In detail, this amounts to the pre-specification of the objective and oped the Navigation Guide (Woodruff and Sutton, 2014), and the US Of- methods of the SR in a written protocol, in which the aim of conducting fice of Health Assessment and Translation (OHAT) at the US National the review is clearly stated as a structured question (for a SR of the ef- Toxicology Program has developed the OHAT Framework for systemat- fects of an intervention or exposure, this can establish a testable hypoth- ically reviewing environmental health research for hazard identification esis or quantitative parameter that is to be estimated), along with the (Rooney et al., 2014). articulation of appropriate methods. The methods specified should in- The two approaches adapt the key elements of SR methods to ques- clude the techniques for identifying literature of potential relevance to tions in environmental health (which is directly relevant to the CRA the research question, the criteria for inclusion of the studies of actual process but does not include assessment of dose–response). Features relevance to the research question, how the internal validity of the that the two approaches have in common include: conducting a SR ac- cording to a pre-specified protocol; the development of a specific research question and use of “PECO” statements (see Box 2)in “Internal validity” is a term used in Cochrane Collaboration guidance on conduct of SRs systematising review objectives and the methods that will be used to specifically intended to supersede the use of terms such as “methodological quality” or answer that question; an approach to appraising the internal validity their equivalents, which are considered ambiguous (Higgins and Green, 2011). The inter- nal validity of a piece of research is appraised in a “risk of bias” assessment. The target of of included studies adapted from the risk of bias appraisal tool devel- the risk of bias assessment is the likelihood, magnitude and direction of systematic error oped by the Cochrane Collaboration (Higgins et al., 2011); an adaptation in the size of an observed effect, as caused by flaws in the design, conduct, analysis and of the GRADE methodology (Atkins et al., 2004) for describing the cer- reporting of a study. Throughout this document, we follow Cochrane Collaboration con- tainty or strength of a body of evidence, incorporating risk of bias ele- ventions in using “internal validity” as a technical term in place of “methodological quality”. ments with other criteria such as for the assessment of relevance or Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 6 P. Whaley et al. / Environment International xxx (2015) xxx–xxx Box 2. The use of PECO statements in the SR process. external validity; and a methodology for combining the results of assessment, concentrations of a chemical in the environment and human and animal research into a statement of confidence about the biota, and the derivation of a No Observed Adverse Effect Level hazard which a chemical poses to health. (NOAEL) or Benchmark Dose Lower 95% confidence limit (BMDL). Other tools are being developed to contribute to the systematic as- European Food Safety Authority (2015c) explores these issues in more sessment of in vivo and ecotoxicity studies which have not been directly detail. derived from Cochrane Collaboration methods. Presented at the Work- Depending on scope, the resources (time and cost) to undertake an shop was SciRAP (Science in Risk Assessment and Policy), a system de- SR can be considerable. Currently there is a lack of empirical evidence veloped to improve the consistency with which the relevance and relating to the resource-effectiveness of SR approaches in CRA and there was a difference of opinion among workshop participants as to reliability of studies are appraised in the context of conducting a chem- ical risk assessment for regulatory purposes. It is also intended to reduce whether the effort required for conducting a SR tends to be under- or the risk of selection bias in the risk assessment process by providing a overestimated. It was suggested that, where effort is likely to be sub- mechanism for including non-standardised study methods yielding po- stantial, efficient use of resources may be achieved by focusing on tentially valuable data (Beronius et al., 2014; SciRAP, 2014). high-value questions developed through initial scoping exercises. For There are a number of other initiatives promoting and developing example, a low-dose adverse effect may be evident in animal models the use of SR methodologies in environmental and chemical risk assess- and supported to some extent by human epidemiology and hence a ment. Participants heard about how the European Food Safety Authority question may be formulated around this initial evidence; there may be is integrating SR methods into its assessments of food and feed safety little point, however, in pursuing a question related to non- (European Food Safety Authority, 2015b, 2015c), and about the UK carcinogenic toxicity in wildlife if a substantial part of the literature Joint Water Evidence Group methods for rapid and systematic assess- points towards that substance being a potential human carcinogen. ments of evidence (Collins et al., 2014). Other coordinated initiatives in- There is also growing interest in rapid reviews, when full SR methods clude the Evidence-Based Toxicology Collaboration (Hoffmann and are considered overly onerous (Collins et al., 2014; Schünemann and Moja, 2015). Hartung, 2006); the Collaboration for Environmental Evidence (Bilotta et al., 2014a; Land et al., 2015); and the Systematic Review Centre for The priorities for expediting the adaptation of SR methods to CRA Laboratory Animal Experimentation (SYRCLE). identified at the Workshop are as follows: 1. The development of a number of prototype CRA-focused SRs to ex- 3.3. Overcoming the challenges in implementing SR methods in CRA plore how readily SR procedures can be integrated into the CRA pro- cess, to: Risk assessment for a chemical or group of chemicals is a multi- a. identify additional methodological challenges in adapting SR faceted process that normally requires consideration of multiple end- methods to the CRA context and develop techniques to address points in relation to a variety of exposure scenarios, integrating evi- them; dence from epidemiological studies, bioassays in animals, mechanistic b. acquire practical experience in managing resources when studies and studies on the distribution and determinants of exposure conducting SRs in CRA, including the conduct of scoping exercises by different pathways and routes. In addition to resolving methodolog- for identifying high-value review questions, the further develop- ical issues relating to underdeveloped methods (e.g. how SR methods ment and/or application of novel “rapid evidence review” methods can be used as part of dose–response assessment or how they can be ap- (UK Civil Service, 2015), and how SR methods can be integrated plied to exposure assessment), it is important to consider how SR into existing regulatory structures such as REACH (see Box 3) should fit into the CRA process. One challenge going forward is to ex- (European Chemicals Agency (2/26/2015)). plore the circumstances in which applying more rigorous SR methods to assess scientific evidence would be warranted, which would require 2. Technical development of SR methodologies for CRA purposes, in insight into the practicality and cost-effectiveness of applying such particular the further advancement of techniques for appraising methods in those situations. and synthesising mechanistic, toxicological and human epidemio- In principle, it should be possible to conduct SRs in any aspect of a logical studies, to include: CRA. Given the success in employing SR methods to support evidence- a. refining tools for more consistent and scientifically robust ap- based practice in healthcare, it is intuitive that SRs could address specific praisal of the internal validity of individual studies included in a questions arising within toxicology, human epidemiology and environ- CRA and the implications for interpretation of their findings; see mental health (e.g. hazard assessment within a CRA) and this view ap- e.g. Bilotta et al. (2014b). This might include further development pears to be gaining momentum within the environmental health and validation of tools such as the SYRCLE methodology for literature. The SR method may also lend itself to answering questions assessing the internal validity of animal studies (Hooijmans et al., concerning e.g. the accuracy of the reported physical-chemical proper- 2014); for SR of observational studies see e.g. Sterne et al. (2014), ties of a substance, doses predicted by quantitative exposure Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 P. Whaley et al. / Environment International xxx (2015) xxx–xxx 7 of SR methods to CRA as a complement to current courses which largely cover SR methods in healthcare. 4. Conclusions While systematic review methods have proven highly influential in healthcare, they have yet to make widespread impact on the process of chemical risk assessment. While there is much promise in the concept of adapting SR methods to CRA to give definitive answers to specified research questions, or to enable identification of the reasons for failure to resolve debate, a number of challenges to implementing SR methods in CRA have been identified. These include particular concerns about ap- proaches to assessing bias and confounding in observational studies, the effort involved in conducting SRs, and the subsequent benefits of conforming to SR standards. Recent experience from both regulatory agencies and academics already yields some clear recommendations which would expedite the wider implementation of SR methods in CRA, potentially increasing the efficiency, transparency and scientific robustness of the CRA process. Disclaimer The views expressed in this manuscript are those of the authors and do not necessarily represent the views or policies of their employers or otherwise affiliated organisations. EA is employed by the European Food Safety Authority (EFSA); however, the present article is published under her sole responsibility and may not be considered as an EFSA scientific output. Box 3. The potential utility of SR methods in application to REACH registrations. Acknowledgements Funding for the workshop was provided through the Economic & So- cial Science Research Council grant “Radical Futures in Social Sciences” the methods employed in the NTP/OHAT and Navigation Guide (Lancaster University) and Lancaster Environment Centre. CH, PW, AR protocols, and the applicability of other assessment methods are grateful to Lancaster University's Faculty of Science & Technology such as SciRAP (Beronius et al., 2014); “Distinguished Visitors” funding programme. The Royal Society of b. the development of tools for the hazard characterisation and expo- Chemistry is acknowledged for generously providing a meeting room, sure assessment components of the CRA process; refreshments and facilitating the workshop proceedings. The PhD stu- c. the further development of software akin to the Cochrane dentship of PW is partly funded through Lancaster Environment Centre. Collaboration's Review Manager (Nordic Cochrane Centre, 2014) The contribution of non-author workshop participants to the develop- and the Systematic Review Data Repository (Ip et al., 2012), and ment of the manuscript is also greatly appreciated. toolssuchasDRAGON(ICF International, 2015) and the Health As- sessment Workspace Collaborative (Rusyn and Shapiro, 2013)to References support extraction, analysis and sharing of data from studies in- cluded in reviews; Aiassa, E., Higgins, J.P.T., Frampton, G.K., Greiner, M., Afonso, A., Amzal, B., et al., 2015. Ap- plicability and feasibility of systematic review for performing evidence-based risk as- sessment in food and feed safety. Crit. Rev. Food Sci. Nutr. 55 (7), 1026–1034. http:// dx.doi.org/10.1080/10408398.2013.769933. 3. The development an empirical evidence base for the different types Antman, E.M., Lau, J., Kupelnick, B., Mosteller, F., Chalmers, T.C., 1992. A comparison of re- of bias that operate in the CRA domain, including their direction sults of meta-analyses of randomized control trials and recommendations of clinical and potential magnitude, and the extent to which any methods experts. Treatments for myocardial infarction. JAMA 268 (2), 240–248. Atkins, D., Best, D., Briss, P.A., Eccles, M., Falck-Ytter, Y., Flottorp, S., et al., 2004. Grading being adopted to address them are appropriate and effective. quality of evidence and strength of recommendations. BMJ (Clinical research Ed.) 4. The development of a recognised “gold standard” for SRs in toxicol- 328 (7454), 1490. http://dx.doi.org/10.1136/bmj.328.7454.1490. ogy and risk assessment equivalent to the Cochrane Collaboration Beronius, A., Molander, L., Rudén, C., Hanberg, A., 2014. Facilitating the use of non- standard in vivo studies in health risk assessment of chemicals: a proposal to improve in evidence-based medicine, to address the growing number of pur- evaluation criteria and reporting. Journal of Applied Toxicology: JAT 34 (6), 607–617. ported SRs of unclear validity which are increasingly prevalent in the http://dx.doi.org/10.1002/jat.2991. environmental health literature. Bilotta, G.S., Milner, A.M., Boyd, I., 2014a. On the use of systematic reviews to inform en- vironmental policies. Environ. Sci. Pol. 42, 67–77. http://dx.doi.org/10.1016/j.envsci. 5. The creation of a climate of constructive discussion that fosters ad- 2014.05.010. vancement of methods whereby chemical risk practitioners, indus- Bilotta, G.S., Milner, A.M., Boyd, I.L., 2014b. Quality assessment tools for evidence try, competent authorities, academic researchers and policy makers from environmental science. Environ. Evid. 3(1),14. http://dx.doi.org/10.1186/ can research, discuss and evaluate SR methods and the potential ad- 2047-2382-3-14. Birnbaum, L.S., Thayer, K.A., Bucher, J.R., Wolfe, M.S., 2013. Implementing systematic re- vantages they can bring. view at the National Toxicology Program: status and next steps. Environ. Health 6. The establishment of a network of scientists and CRA practitioners to Perspect. 121 (4), A108–A109. http://dx.doi.org/10.1289/ehp.1306711. pursue research into and discussion of SR methodologies and facili- Campbell Collaboration, 2015. The Campbell Collaboration. Available online at http:// www.campbellcollaboration.org/, accessed 6/13/2015. tate their implementation. Caveman, 2000. The invited review — or, my field, from my standpoint, written by me 7. The implementation of training programmes for risk assessment using only my data and my ideas, and citing only my publications. J. Cell Sci. 113 practitioners and stakeholders, focusing specifically on application (Pt 18), 3125–3126. Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 8 P. Whaley et al. / Environment International xxx (2015) xxx–xxx Chalmers, I., Glasziou, P., 2009. Avoidable waste in the production and reporting of re- Lau, J., Rothstein, H.R., Stewart, G.B., 2013. History & progress of meta-analysis. In: search evidence. Lancet 374 (9683), 86–89. http://dx.doi.org/10.1016/S0140-6736 Koricheva, J., Gurevitch, J., Mengersen, K. (Eds.), Handbook of Meta-analysis in Ecol- (09)60329-9. ogy and Evolution. Princeton University Press, Princeton Chapter 25. Chalmers, I., Hedges, L.V., Cooper, H., 2002. A brief history of research synthesis. Evalua- Macleod, M.R., Ebrahim, S., Roberts, I., 2005. Surveying the literature from animal exper- tion & the Health Professions 25 (1), 12–37. http://dx.doi.org/10.1177/ iments: systematic review and meta-analysis are important contributions. BMJ 331 0163278702025001003. (7508), 110. http://dx.doi.org/10.1136/bmj.331.7508.110-b. Clarke, M., Brice, A., Chalmers, I., 2014. Accumulating research: a systematic account of Mignini, L.E., Khan, K.S., 2006. Methodological quality of systematic reviews of animal how cumulative meta-analyses would have provided knowledge, improved health, studies: a survey of reviews of basic research. BMC Med. Res. Methodol. 6, 10. reduced harm and saved resources. PLoS ONE 9 (7), e102670. http://dx.doi.org/10. http://dx.doi.org/10.1186/1471-2288-6-10. 1371/journal.pone.0102670. Mulrow, C.D., 1987. The medical review article: state of the science. Ann. Intern. Med. 106 Collaboration for Environmental Evidence (2015): The Collaboration for Environmental Evi- (3), 485–488. dence. Available online at http://www.environmentalevidence.org/, accessed 6/13/2015. National Food Institute, Denmark (2015): Evaluation of EFSA's new scientificopinion on Collins, A., Miller, J., Coughlin, D., Kirk, S., 2014. The Production of Quick Scoping Reviews bisphenol A. Søborg, Denmark (REG-no. DK 30 06 09 46). Available online at http:// and Rapid Evidence Assessments: A How to Guide (Beta Version 2). Joint Water Ev- www.food.dtu.dk/english/~/media/Institutter/Foedevareinstituttet/Publikationer/ idence Group. Available online at https://sbri.innovateuk.org/documents/3058188/ Pub-2015/Evaluation_BisphenolA.ashx?la=da. 3918930/The+Production+of+QSRs+and+REAs-+A+How+to+guide.pdf/ Nordic Cochrane Centre (2014): Review Manager (RevMan). Version 5.3: Cochrane Col- 45975020-be7d-4788-b74b-f3b6ed32c73a. laboration. Available online at http://tech.cochrane.org/revman, accessed 6/18/2015. European Chemicals Agency (2/26/2015), Evaluation under REACH Progress Report 2014. Plastics Europe (1/15/2015): French ban on the use of Bisphenol A (BPA) in food contact: Available online at: http://echa.europa.eu/documents/10162/13628/evaluation_ In conflict with European law and risk assessment — severe distortion of the market — report_2014_en.pdf. Accessed 4/11/2015. no safety benefit for consumers. Jasmin Bird. Available online at http://www. European Commission (1/28/2011): Directive 2011/8/EU of 28 January 2011 amending bisphenol-a-europe.org/uploads/Modules/Mediaroom/stm_re_french-bpa-ban- Directive 2002/72/EC as regards the restriction of use of Bisphenol A in plastic infant being-enforced-01-01-2015.pdf, accessed 6/15/2015. feeding bottles, Directive 2011/8/EU. In : Official Journal of the European Union. Rooney, A.A., Boyles, A.L., Wolfe, M.S., Bucher, J.R., Thayer, K.A., 2014. Systematic review Available online at http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L: and evidence integration for literature-based environmental health science assess- 2011:026:0011:0014:EN:PDF, accessed 6/15/2015. ments. Environ. Health Perspect. 122 (7), 711–718. http://dx.doi.org/10.1289/ehp. European Commission (5/31/2011): Bisphenol A: EU ban on baby bottles to enter into 1307972. force tomorrow. Brussels. Available online at http://europa.eu/rapid/press-release_ Rusyn, I.; Shapiro, A. (2013): Health Assessment Workspace Collaborative (HAWC). Ver- IP-11-664_en.htm, accessed 2/17/2015. sion Solid Hammer: UNC-CH Software. Available online at https://hawcproject.org/, European Food Safety Authority (2010): Application of systematic review methodology accessed 8/4/2015. to food and feed safety assessments to support decision making. EFSA Journal Salman, R.A.-S., Beller, E., Kagan, J., Hemminki, E., Phillips, R.S., Savulescu, J., et al., 2014. 2010; 8(6):1637. DOI: http://dx.doi.org/10.2903/j.efsa.2010.1637. Increasing value and reducing waste in biomedical research regulation and manage- European Food Safety Authority (2015a): No consumer health risk from bisphenol A ex- ment. Lancet 383 (9912), 176–185. http://dx.doi.org/10.1016/S0140-6736(13) posure. Press Release 21 Jan 2015. Parma. Available online at http://www.efsa. 62297-7. europa.eu/en/press/news/150121.htm, accessed 2/18/2015. Schünemann, H.J., Moja, L., 2015. Reviews: Rapid! Rapid! Rapid! … and systematic. Sys- European Food Safety Authority (2015b): Outcome of the targeted consultation of the tematic Reviews 4, 4. http://dx.doi.org/10.1186/2046-4053-4-4. EFSA Journal editorial on increasing openness, robustness and transparency of scien- SciRAP (2014): Science in risk assessment and policy. Department of Applied Environ- tific assessments. Available online at http://www.efsa.europa.eu/en/supporting/pub/ mental Science at Stockholm University; Institute of Environmental Medicine at 785e.htm, accessed 8/4/2015. Karolinska Institutet in Stockholm; MistraPharma. Available online at http://www. European Food Safety Authority, 2015c. Principles and process for dealing with data and scirap.org/, accessed 3/11/2015. evidence in scientific assessments. EFSA Journal 13 (5), 4121. http://dx.doi.org/10. Sena, E.S., Currie, G.L., McCann, S.K., Macleod, M.R., Howells, D.W., 2014. Systematic re- 2903/j.efsa.2015.4121. views and meta-analysis of preclinical studies: why perform them and how to ap- France (12/24/2012): LOI no. 2012–1442 du 24 décembre 2012 visant à la suspension de praise them critically. J. Cereb. Blood Flow Metab. 34 (5), 737–742. http://dx.doi. la fabrication, de l'importation, de l'exportation et de la mise sur le marché de tout org/10.1038/jcbfm.2014.28. conditionnement à vocation alimentaire contenant du bisphénol A. Legifrance.gouv. Silbergeld, E., Scherer, R.W., 2013. Evidence-based toxicology: strait is the gate, but the fr. Available online at http://legifrance.gouv.fr/affichTexte.do;jsessionid= road is worth taking. ALTEX 30 (1), 67–73. F6553AACC19D178279D8DF154EAC8558.tpdila17v_1?cidTexte= Sterne, J.A.C.; Higgins, J.P.T.; Reeves, B.C. (2014): A Cochrane Risk of Bias Assessment Tool for JORFTEXT000026830015, accessed 6/15/2015. Non-Randomized Studies of Interventions (ACROBAT-NRSI). The Cochrane Collabora- French Agency for Food, Environmental and Occupational Health & Safety (2013): tion. Available online at https://sites.google.com/site/riskofbiastool/, accessed 9/29/2014. Bisphenol A: ANSES demonstrates potential health risks and confirms the need to re- Stewart, G., 2010. Meta-analysis in applied ecology. Biol. Lett. 6 (1), 78–81. http://dx.doi. duce exposure. Available online at https://www.anses.fr/en/content/bisphenol- org/10.1098/rsbl.2009.0546. anses-demonstrates-potential-health-risks-and-confirms-need-reduce-exposure. Stewart, G.B., Schmid, C.H., 2015. Lessons from meta-analysis in ecology and evolution: French Agency for Food, Environmental and Occupational Health & Safety (4/7/2014): the need for trans-disciplinary evidence synthesis methodologies. Research Synthesis Bisphenol A: ANSES publishes its comments in response to the EFSA draft opinion for Methods 6 (2), 109–110. http://dx.doi.org/10.1002/jrsm.1152. consultation. Available online at https://www.anses.fr/en/content/bisphenol-anses- Tošenovský, E. (2014): Question for written answer to the Commission, Rule 130. publishes-its-comments-response-efsa-draft-opinion-consultation, accessed 2/18/2015. European Parliament, Parliamentary questions, P-008546/2014, Subject: possible Higgins, J P. T.; Green, S. (Eds.) (2011): Cochrane handbook for systematic reviews of in- negative impact on the internal market of measures concerning BPA adopted by terventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Avail- the French authorities. 30 October 2014. Available online at http://www.europarl. able online at http://handbook.cochrane.org/, accessed 2/18/2015. europa.eu/sides/getDoc.do?type=WQ&reference=P-2014-008546&language=EN, Higgins, J.P.T., Altman, D.G., Gøtzsche, P.C., Jüni, P., Moher, D., Oxman, A.D., et al., 2011. The accessed 6/15/2015. Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 343, Tošenovský, E. (2015): Question for written answer to the Commission, Rule 130. d5928. http://dx.doi.org/10.1136/bmj.d5928. European Parliament, Parliamentary questions, E-004315-15, Subject: measures Hoffmann, S., Hartung, T., 2006. Toward an evidence-based toxicology. Hum. Exp. Toxicol. concerning Bisphenol A. 17 March 2015. Available online at http://www.europarl. 25 (9), 497–513. http://dx.doi.org/10.1191/0960327106het648oa. europa.eu/sides/getDoc.do?pubRef=-//EP//TEXT+WQ+E-2015-004315+0+DOC Hooijmans, C.R.,Rovers, M.,de Vries,R.B., Leenaars, M., Ritskes-Hoitinga, M., 2012. An initia- +XML+V0//EN&language=en, accessed 6/15/2015. tive to facilitate well-informed decision-making in laboratory animal research: report of UK Civil Service (2015): What is a rapid evidence assessment? Available online at http:// the First International Symposium on Systematic Reviews in Laboratory Animal Science. www.civilservice.gov.uk/networks/gsr/resources-and-guidance/rapid-evidence- Lab. Anim. 46 (4), 356–357. http://dx.doi.org/10.1258/la.2012.012052. assessment/what-is, accessed 6/13/2015. Hooijmans, C.R., Rovers, M.M., de Vries, R.B.M., Leenaars, M., Ritskes-Hoitinga, M., US Environmental Protection Agency (2013): Process for developing IRIS health assess- Langendam, M.W., 2014. SYRCLE's risk of bias tool for animal studies. BMC Med. ments. Available online at http://www.epa.gov/IRIS/process.htm,accessed 6/16/ Res. Methodol. 14, 43. http://dx.doi.org/10.1186/1471-2288-14-43. 2015. ICF International (2015): DRAGON: an online tool for systematic review. Available online US Food and Drug Administration (2014): Bisphenol A (BPA): use in food contact appli- at http://www.icfi.com/insights/products-and-tools/dragon-online-tool-systematic- cation. Update on Bisphenol A (BPA) for Use in Food Contact Applications. Available review, accessed 8/4/2015. online at http://www.fda.gov/NewsEvents/PublicHealthFocus/ucm064437.htm. International Agency for Research on Cancer (2006): Preamble to the IARC Monographs. US Institute of Education Sciences (2015): What works clearinghouse. Available online at Lyon, France. Available online at http://monographs.iarc.fr/ENG/Preamble/index.php, http://ies.ed.gov/ncee/wwc/default.aspx, accessed 6/13/2015. accessed 9/10/2015. US National Research Council, 2014a. A Framework to Guide Selection of Chemical Alter- Ip, S., Hadar, N., Keefe, S., Parkin, C., Iovin, R., Balk, E.M., Lau, J., 2012. A Web-based archive natives. The National Academies Press, Washington, D.C. of systematic review data. Systematic Reviews 1, 15. http://dx.doi.org/10.1186/2046- US National Research Council, 2014b. Review of EPA's Integrated Risk Information System 4053-1-15. (IRIS) Process. The National Academies Press, Washington, D.C. Land, M., de Wit, C.A., Cousins, I.T., Herzke, D., Johansson, J., Martin, J.W., 2015. What is the US National Toxicology Panel (2015): Handbook for Conducting a Literature-Based Health effect of phasing out long-chain per- and polyfluoroalkyl substances on the concentra- Assessment Using OHAT Approach for Systematic Review and Evidence Integration. tions of perfluoroalkyl acids and their precursors in the environment? A systematic re- Available online at http://ntp.niehs.nih.gov/ntp/ohat/pubs/handbookjan2015_508. view protocol. Environ. Evid. 4(1), 3. http://dx.doi.org/10.1186/2047-2382-4-3. pdf, accessed 1/13/2015. Lau, J., Ioannidis, J.P.A., Schmid, C.H., 1998. Summing up evidence. One answer is not al- van Luijk, J., Bakker, B., Rovers, M.M., Ritskes-Hoitinga, M., de Vries, R.B.M., Leenaars, M., ways enough. Lancet 351 (9096), 123–127. http://dx.doi.org/10.1016/S0140-6736 2014. Systematic reviews of animal studies; missing link in translational research? (97)08468-7. PLoS ONE 9 (3), e89981. http://dx.doi.org/10.1371/journal.pone.0089981. Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 P. Whaley et al. / Environment International xxx (2015) xxx–xxx 9 Vandenberg, L.N., Ehrlich, S., Belcher, S.M., Ben-Jonathan, N., Dolinoy, D.C., Hugo, E.R., et better health outcomes. Environ. Health Perspect. 122 (10), 1007–1014. http://dx. al., 2014. Low dose effects of bisphenol A. Endocrine Disruptors 1 (1), e26490. doi.org/10.1289/ehp.1307175. http://dx.doi.org/10.4161/endo.26490. Zoeller, R.T., Bergman, Å., Becher, G., Bjerregaard, P., Bornman, R., Brandt, I., et al., 2015. A Woodruff, T.J., Sutton, P., 2014. The Navigation Guide systematic review methodology: a path forward in the debate over health impacts of endocrine disrupting chemicals. rigorous and transparent method for translating environmental health science into Environ. Heal. 14, 118. http://dx.doi.org/10.1186/1476-069X-13-118. Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Environment International Unpaywall

Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations

Download PDF
9 pages

Loading next page...
 
/lp/unpaywall/implementing-systematic-review-techniques-in-chemical-risk-assessment-doWZRtzprE

References (71)

Publisher
Unpaywall
ISSN
0160-4120
DOI
10.1016/j.envint.2015.11.002
Publisher site
See Article on Publisher Site

Abstract

EI-03128; No of Pages 9 Environment International xxx (2015) xxx–xxx Contents lists available at ScienceDirect Environment International journal homepage: www.elsevier.com/locate/envint Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations a a, b d c e Paul Whaley , Crispin Halsall , Marlene Ågerstrand ,Elisa Aiassa , Diane Benford , Gary Bilotta , f w n g h David Coggon , Chris Collins , Ciara Dempsey , Raquel Duarte-Davidson , Rex FitzGerald , x i j k l m Malyka Galay-Burgos , David Gee , Sebastian Hoffmann , Juleen Lam , Toby Lasserson , Len Levy , n o i p q Steven Lipworth , Sarah Mackenzie Ross , Olwenn Martin , Catherine Meads , Monika Meyer-Baron , r s t u v n James Miller , Camilla Pease , Andrew Rooney ,Alison Sapiets , Gavin Stewart , David Taylor Lancaster Environment Centre, Lancaster University, Lancaster LA1 4YQ, UK Department of Environmental Science and Analytical Chemistry, Stockholm University, SE-106 91, Stockholm, Sweden Food Standards Agency, Aviation House, 125 Kingsway, London WC2B 6NH, UK Assessment and Methodological Support Unit, European Food Safety Authority, Via Carlo Magno 1/a 43126, Parma, Italy Aquatic Research Centre, University of Brighton, Lewes Road, Brighton BN2 4GJ, UK MRC Lifecourse Epidemiology Unit, University of Southampton, MRC Lifecourse Epidemiology Unit, Southampton General Hospital, Southampton SO16 6YD, UK Centre for Radiation, Chemicals and Environmental Hazards, Public Health England, Harwell Science and Innovation Campus, Didcot, Oxfordshire OX11 0RQ, UK Swiss Centre for Applied Human Toxicology, University of Basel, Missionsstrasse 64, 4055 Basel, Switzerland Institute for the Environment, Health and Societies, Brunel University London, Kingston Lane, Uxbridge UB8 3PH, UK Evidence-Based Toxicology Collaboration (EBTC), Stembergring 15, 33106 Paderborn, Germany University of California San Francisco, Program on Reproductive Health and the Environment, San Francisco, CA, USA Cochrane Editorial Unit, Cochrane Central Executive, St Albans House, 57-9 Haymarket, London SW1Y 4QX, UK Institute of Environment, Health, Risks and Futures, School of Energy, Environment and Agrifood, Cranfield University, Cranfield, Bedfordshire MK43 0AL, UK Royal Society of Chemistry, Burlington House, Piccadilly, London W1J 0BA, UK Research Department of Clinical, Educational and Health Psychology, University College London, Gower Street, London WC1E 6BT, UK Health Economics Research Group, Brunel University London, Kingston Lane, Uxbridge UB8 3PH, UK Leibniz Research Centre for Working Environment and Human Factors (IfADo), Neurobehavioural Toxicology, Ardeystr 67, D-44139 Dortmund, Germany Centre for Ecology and Hydrology, Wallingford, Oxfordshire 0X10 8BB, UK Ramboll Environ, 1 Broad Gate, The Headrow, Leeds LS1 8EQ, UK National Institute of Environmental Sciences (NIEHS), National Institutes of Health (NIH), Department of Health and Human Services (DHHS), Research Triangle Park, NC, USA Syngenta Ltd., Jealott's Hill International Research Centre, Bracknell RG42 6EY, UK Centre for Rural Economy, School of Agriculture, Food and Rural Development, University of Newcastle upon Tyne, UK Department of Geography and Environmental Science, School of Archaeology, Geography and Environmental Science, University of Reading, Reading, RG6 6DW, United Kingdom European Centre for Ecotoxicology and Toxicology of Chemicals (ECETOC), Avenue Edmond Van Nieuwenhuyse 2 Bte 8B-1160 Brussels, Belgium article i nfo abstract Article history: Systematic review (SR) is a rigorous, protocol-driven approach designed to minimise error and bias when Received 8 August 2015 summarising the body of research evidence relevant to a specificscientific question. Taking as a comparator the Accepted 2 November 2015 use of SR in synthesising research in healthcare, we argue that SR methods could also pave the way for a “step Available online xxxx change” in the transparency, objectivity and communication of chemical risk assessments (CRA) in Europe and else- where. We suggest that current controversies around the safety of certain chemicals are partly due to limitations in Keywords: current CRA procedures which have contributed to ambiguity about the health risks posed by these substances. We Risk assessment present an overview of how SR methods can be applied to the assessment of risks from chemicals, and indicate how Research synthesis challenges in adapting SR methods from healthcare research to the CRA context might be overcome. Regarding the Environment Chemicals latter, we report the outcomes from a workshop exploring how to increase uptake of SR methods, attended by ex- Systematic review perts representing a wide range of fields related to chemical toxicology, risk analysis and SR. Priorities which were Toxicology identified include: the conduct of CRA-focused prototype SRs; the development of a recognised standard of reporting and conduct for SRs in toxicology and CRA; and establishing a network to facilitate research, communica- tion and training in SR methods. We see this paper as a milestone in the creation of a research climate that fosters communication between experts in CRA and SR and facilitates wider uptake of SR methods into CRA. © 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). ⁎ Corresponding author. E-mail address: [email protected] (C. Halsall). http://dx.doi.org/10.1016/j.envint.2015.11.002 0160-4120/© 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 2 P. Whaley et al. / Environment International xxx (2015) xxx–xxx 1. Introduction considered is usually much wider than in the assessment of healthcare interventions. Thus, when the various types of toxicological research Systematic review (SR) is a rigorous, protocol-driven approach to are combined into a single overall conclusion about the health risks minimising error and bias in the aggregation and appraisal of evidence posed by a chemical exposure, reviewers are challenged with integrat- relevant to answering a research question. SR techniques were initially ing the results from a broad and heterogeneous evidence base. developed in the fields of psychology, social science and health care and In spite of these differences, there is reason for thinking that SR have, since the 1980s, provided a valuable tool for evidence-informed methods can be applied successfully to CRA. For example, techniques decision-making across many domains (Lau et al., 2013). In medicine, for aggregating the results of different study types are already addressed SRs have provided a valuable response to the need for consistent, trans- in various frameworks currently in use in toxicology. These include: In- parent and scientifically-robust interpretations of the results of increas- ternational Agency of Research on Cancer (IARC) Monographs ing numbers of often conflicting studies of the efficacy of healthcare (International Agency for Research on Cancer, 2006); the Navigation interventions. SRs have taken on an increasingly fundamental role Guide (Woodruff and Sutton, 2014); and the US Office for Health As- both in supporting decision-making in healthcare and, by channelling sessment and Translation (OHAT) (Rooney et al., 2014; US National Tox- resources towards questions for which the answers are not yet icology Panel, 2015) – though it should be noted that none of these known, reducing waste in research (Chalmers and Glasziou, 2009; approaches have yet applied SR methods to the exposure assessment Salman et al., 2014). It is now accepted practice in healthcare to use component of CRA. Heterogeneous sources of evidence are a familiar SR methods to assess evidence not only for the efficacy of interventions, challenge in all domains including clinical medicine (Lau et al., 1998), but also on diagnostic tests, prognostics and adverse outcomes. and SR of observational studies has a crucial role in identifying compli- The extension of SR techniques to other fields is based on a mutual cations and side-effects of healthcare interventions (Sterne et al., 2014; need across disciplines to make the best use of existing evidence Higgins and Green, 2011). The need for SR of pre-clinical animal trials of when making decisions, a move for which momentum has been grow- healthcare interventions, in order to better anticipate benefits and ing for several decades. For example, the What Works Clearinghouse harms to humans, is another area in which methods being developed was established in 2002 to apply SR techniques in support of and implemented by a number of groups including SYRCLE American educational policy (US Institute of Education Sciences, (Hooijmans et al., 2012; van Luijk et al., 2014) and CAMARADES 2015), and in 2000 the international Campbell Collaboration research (Macleod et al., 2005; Sena et al., 2014). (Stewart and Schmid, 2015) network was convened to undertake and disseminate systematic re- argue that research synthesis methods (including systematic review) views on the effects of social interventions in diverse fields such as are generic and applicable to any domain if appropriately crime and justice, education, international development and social wel- contextualised. fare (Campbell Collaboration, 2015). Meta-analysis and SR in ecology Given the sometimes controversial outcomes of CRAs and the grow- have contributed to evidence-based environmental policy since the ing public and media profile of the risks that chemicals may pose to mid-1990s (Stewart, 2010); more recently, the Collaboration for Envi- humans and the environment, SR is increasingly viewed as a potentially ronmental Evidence (CEE) has been established to encourage conduct powerful technique in assessing and communicating how likely it is that of SRs on a wide range of environmental topics (Collaboration for a chemical will cause harm. SR methods add transparency, rigour and Environmental Evidence, 2015). objectivity to the process of collecting the most relevant scientificevi- The potential advantages of adapting SR methodology to the field of dence with which to inform policy discussions and could provide a crit- chemical risk assessment (CRA) have also been recognised, with multi- ical tool for organising and appraising the evidence on which chemical ple research groups and organisations either developing and adopting policy decisions are based. (Woodruff and Sutton, 2014; Birnbaum et al., 2013; European Food Consequently, in November 2014 a group of 35 scientists and re- Safety Authority, 2010; Rooney et al., 2014; Aiassa et al., 2015)or searchers from the fields of medicine, toxicology, epidemiology, envi- recommending (US National Research Council, 2014a, 2014b; US Envi- ronmental chemistry, ecology, risk assessment, risk management and ronmental Protection Agency, 2013; Silbergeld and Scherer, 2013; Hoff- SR participated in a one-day workshop to consider the application of mann and Hartung, 2006; Zoeller et al., 2015) the use of SR methods for SR in CRA. The purpose was three-fold: evaluating the association between health effects and chemical expo- 1. Identify from expert practitioners in risk assessment and SR the ob- sures to inform decision-making. There are, however, a number of stacles, in terms of practical challenges and knowledge gaps, to recognised challenges in extending SR methods to CRA, many of implementing SR methods in CRA; which derive from key differences in the evidence base between the healthcare and toxicological sciences. 2. Develop a “roadmap” for overcoming those obstacles and expediting SRs in medicine often focus on direct evidence for benefits and ad- the implementation of SR methods, where appropriate, by the vari- verse effects of healthcare interventions derived from randomised con- ous stakeholders involved in CRA; trolled trials (RCTs) in humans. The evidence base for CRA is generally 3. Establish the foundations of a network to co-ordinate research and more complex, with a need to extrapolate from investigations in ani- activities relating to the implementation of SR methods in CRA. The mals, in vitro and in silico, and then to synthesise findings with those aim would be to support best practise in the application of SR tech- from human studies if available. Furthermore, the human data tend to niques and promote the wider adoption of SR in CRA, both in come from observational studies with greater and more varied potential Europe and elsewhere. for bias and confounding than RCTs, and the range of outcomes to be Participants heard seven presentations about recent developments in SR methods, their application to the risk assessment process, and It is worth drawing a distinction between three sources of bias in the review process. their potential value to policy-makers. There were two break-out ses- There is potential for bias in the conduct of a review (e.g. because of inappropriate sions in which participants were divided into three facilitated groups, methods for identifying and selecting evidence for inclusion in the review); bias because firstly to discuss challenges to implementing SR methods in CRA, and the material available for the review is not representative of the evidence base as a whole then to suggest ways in which the obstacles could be overcome. These (due to selective publication); and bias arising from flaws in the design, conduct, analysis and reporting of individual studies included in the review that can cause the effect of an ideas were discussed in plenary before being summarised, circulated intervention or exposure to be systematically under- or over-estimated. One of the major for comment, and then published in this paper. The Workshop was con- functions of SRs is to minimise bias in the conduct of a review and, as far as possible, to en- ducted under the “Chatham House Rule” such that participants were sure that potential bias from selective publication and methodological flaws in the evi- free to refer to the information presented and discussed, provided dence are properly taken into account when drawing conclusions in response to a research question. they did not attribute it to identifiable individuals or organisations. Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 P. Whaley et al. / Environment International xxx (2015) xxx–xxx 3 The purpose of this overview paper is to present the rationale for ex- challenging to distinguish which opinion is likely to represent the ploring the application of SR methods to CRA, the various experts' views most valid synthesis of the totality of available evidence. on the challenges to implementing SR methods in CRA, and their sug- A recent illustrative example (see Box 1) of when expert scientists gestions for overcoming them. The remaining goals of the meeting are and reputable organisations have come to apparently contradictory ongoing work, including the development of the roadmap concept for conclusions about the likelihood of a chemical causing harm is the publication and the establishment of a network for supporting the use case of bisphenol-A (BPA). BPA is a monomer used in the manufacture of SR in CRA. of the resinous linings of tin cans and other food contact materials such as polycarbonate drinks bottles. It has been banned from use in infant-feed bottles across the EU (European Commission, 1/28/2011) 2. The appeal of SR methods in CRA because of “uncertainties concerning the effect of the exposure of in- fants to Bisphenol A” (European Commission, 5/31/2011b). Chemical risk assessment is a multi-step process leading to a quanti- The European Food Safety Authority (EFSA) considers that current tative characterisation of risk, which can then be used to inform the levels of exposure to BPA present a low risk of harm to the public management of chemical substances so as to ensure that any risks to (European Food Safety Authority, 2015a). The French food regulator human health or the environment are managed optimally. CRAs entail ANSES takes a seemingly different stance on the risks to health posed four fundamental steps: hazard identification; hazard characterisation by BPA (French Agency for Food, Environmental and Occupational (often a dose–response assessment); exposure assessment; and risk Health, and Safety, 4/7/2014), determining there to be a “potential risk characterisation (see Fig. 1). These steps draw on various fields of scien- to the unborn children of exposed pregnant women”. On this basis, tific research including environmental chemistry, toxicology ANSES has proposed classifying BPA as toxic to reproduction in humans (encompassing in vivo, in vitro, ecotoxicological and in silico methods), (French Agency for Food, Environmental and Occupational Health, and ecotoxicology, human epidemiology, and mathematical modelling. Safety, 2013), a proposal which has contributed to the French authori- There are many ways in which errors can occur in the interpretation ties' decision to implement an outright ban on BPA in all food packaging of evidence from these varied disciplines, including failure to consider materials (France, 12/24/2012). While the ban has been challenged by all relevant data, failure to allow appropriately for the strengths and lim- some stakeholders as being disproportionate under EU law itations of individual studies, and over- or underestimating the rele- (Tošenovský, 2014, 2015; Plastics Europe, 2015 ), the Danish National vance of experimental models to real-world scenarios (to name a Food Institute has argued that EFSA has overestimated the safe daily ex- few). Whether the appraisal of evidence is based on objective processes, posure to BPA and that some populations are exposed to BPA at levels or on subjective expert judgement and opinion, may also be an impor- higher than can be considered safe (National Food Institute, Denmark, tant factor in accurate interpretation of evidence: the assessment pro- 2015); a view reflected in the conclusions of some researchers, e.g. cess always requires input from technical experts, which inevitably (Vandenberg et al., 2014) but not others, e.g. (US Food and Drug brings an element of subjectivity to the interpretation of the scientific Administration, 2014). evidence. Different experts may have varying degrees of practical and The example of BPA illustrates the challenges in reaching consensus cognitive access to relevant information, place differing weight on indi- even when interpreting the same evidence base regarding the potential vidual studies and/or strands of evidence that they review and, when toxicity of chemical exposures, either in terms of what is known and working in committee, may be more or less influenced by dominant what is uncertain about the risks to health posed by BPA, and/or what personalities. This can result in misleading conclusions in which the po- response is appropriate to managing those risks and uncertainties. It tential for health risks is overlooked, underestimated or overstated. Fur- also shows how, in the absence of that consensus, there is a danger thermore, if the factors determining their assessment of evidence are that policy on BPA may become disconnected from the evidence base, undocumented, when expert opinions are in conflict it can be very either risking harm to health through continued exposure or incurring Fig. 1. An overview to the chemical risk assessment (CRA) process, whereby risk is a function of hazard and exposure. While SR methods could in principle be applied to all steps of the CRA process, it is the view of the workshop participants that up to this point in time most attention has been focused on the hazard identification and hazard characterisation steps. There are issues around conducting a systematic review for exposure assessment which were not discussed at the workshop, such as the requirement for a very different tool for assessing risk of bias in exposure studies which may necessitate specialised knowledge of analytical/environmental chemistry. Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 4 P. Whaley et al. / Environment International xxx (2015) xxx–xxx Box 1. Examples of conflicting opinions from scientists and government agencies about the risks to health posed by bisphenol-A at current exposure levels. unnecessary economic costs through restricting the use of a chemical 6. Reproducibility, in that the conclusions of the SR process when ap- which is in fact sufficiently safe. It also suggests that if the reasons for plied to the same question and data should ideally produce the disagreement about health risks posed by a chemical are not accessible same answer even when undertaken by different individuals (also to various stakeholders in the debate, it then becomes much more diffi- described as “consistency”). In practise, different experts may reach cult for regulators to credibly resolve controversies about chemical difference conclusions because they will not all make the same safety, potentially undermining their authority in the long term. value judgments about the scope, quality and interpretation of evi- This example highlights the potential for differences in the interpre- dence. Therefore, the process should be sufficiently rigorous that it tation of evidence when assessing chemical toxicity and the need for a is highly likely that scientific judgement would result in the same process that is not only scientifically robust but also transparent, so conclusion independent of the experts involved, and as a minimum that the reasons for any disagreement can be readily identified – includ- the SR process should render transparent the reasons for all ing giving stakeholders greater opportunity to understand when differ- conclusions. ences in policy stem from divergent assessments of risk, and when they stem from divergent opinions as to how those risks are best managed. It It may be perceived that the value of SR methods lies in their provi- also suggests the importance of the following characteristics in risk as- sion of unequivocal assessments of whether or not a chemical will in- sessments that are used to inform risk management decisions: duce specific harm to humans and/or wildlife in given circumstances. In practise, however, this will happen only if the evidence base is suffi- 1. Transparency, in that the basis for the conclusions of the risk assess- ciently extensive, there is unanimity in identification of the problem and ment should be clear (otherwise they may not be trusted and errors in assessment of the quality of the evidence base, and also how the ev- may go undetected). idence is to be interpreted in answering the review question (without 2. Validity, in that CRAs should be sufficiently (though not necessarily this, SRs will also produce different results). Often, the consensus and/ maximally) scientifically robust in their methodology and accurate or information may be relatively limited; in such circumstances, a SR in their estimation of risks and characterisation of attendant uncer- will instead clearly state the limitations of the available data and conse- tainties as to optimise the decisions that must be made in risk quent uncertainties. The value here is in the provision of a comprehen- management. sive and transparent assessment of what is not known and insight into 3. Confidence, providing the user with a clear statement as to the overall the drivers of divergent opinion. From a research perspective, this yields strength of evidence for the conclusions reached and a characterisa- valuable information about how research limitations and knowledge tion of the utility of the evidence for decision-making (e.g. “appropri- gaps contribute to ongoing uncertainty about environmental and health ate for hazard identification but inappropriate for identification of a risks, allowing the subsequent efforts of researchers to be more clearly reference dose”). focused. From a policy perspective, SRs offer a transparent explanation 4. Utility, in that the output of the risk assessment should be in a form as to why there are differences in opinion which can then be communi- that is convenient and intelligible to those who will use it (outputs cated to stakeholders. that are too detailed and complex to validate and readily compre- Overall, SR contributes to achieving consensus not by eliminating hend lead to inefficiency and possibly erroneous decisions). expert judgement, nor by eliminating conflicting opinions about 5. Efficiency, providing a clear justification of the choice of research whether a compound should be banned (for example), but by providing question in the context of efficiently solving a CRA problem. Re- a robust, systematic and transparent framework for reviewing evidence sources for CRA are often limited and it is wasteful to expend unnec- of risks, such that when there is disagreement, the reasons for it are essary effort on aspects of an assessment that will not be critical to clearly visible and the relative merits of differing opinions can be appraised. In this way, it may help to resolve controversies in the decision-making (although for the purposes of transparency and va- lidity, the reasons for focusing on a particular outcome or otherwise interpretation of the science which informs the risk management restricting the evaluation should be explained). process. Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 P. Whaley et al. / Environment International xxx (2015) xxx–xxx 5 3. SR and its application to CRA included studies will be appraised, and the analytical techniques used for combining the results of the included studies. The purposes of the 3.1. Traditional vs. SR methods protocol are to discourage ad-hoc changes to methodology during the review process which may introduce bias, to allow any justifiable meth- SR methods are often contrasted with “traditional”, non-systematic odological changes to be tracked, and also to allow peer-review of the narrative approaches to describing what is and is not already known work that it is proposed, to help ensure the utility and validity of its ob- in relation to a research question. In reality, the distinction between sys- jectives and methods. tematic and narrative review is a crude one, with narrative reviews The final SR itself consists of a statement of the objective, the search encompassing a number of different approaches to reviewing evidence, method, the criteria for including relevant studies for analysis, and the from the caricature of one researcher writing about “my field, from my results of the appraisal of internal validity of the included studies, e.g. standpoint […] using only my data and my ideas, and citing only my implemented as a “risk of bias” assessment in Cochrane Reviews of publications” (Caveman, 2000), to thorough narrative critiques of com- randomised trials (Higgins et al., 2011). The evidence is then synthe- prehensively identified evidence relevant to answering an explicitly ar- sised using statistical meta-analytical techniques, narrative methods ticulated question, as conducted by organisations such as IARC or both (depending on the extent to which meta-analysis is possible) (International Agency for Research on Cancer, 2006). into an overall answer to the research question. An assessment is then Nonetheless, it is worth noting that only relatively recently has it made of the strength of the evidence supporting the answer; in been recognised that traditional narrative reviews are, to varying de- Cochrane reviews, this typically follows the GRADE methodology grees, vulnerable to a range of methodological shortcomings which (Atkins et al., 2004), taking into account overall features of the evidence are likely to bias their summarisation of the evidence base (Chalmers base including risk of bias across the included studies, publication bias et al., 2002). These include selective rather than comprehensive re- in the evidence base, external validity or applicability of the evidence trieval of evidence relevant to the review topic, inconsistent interpreta- to the population of interest, heterogeneity of the evidence, and the tion of the impact of methodological shortcomings on the validity of overall precision of the evidence. This is finally followed by a concluding included studies, and even an absence of clear review objectives or con- interpretation of what the SR as a whole determines is and is not known clusions which are drawn directly from the strengths and limitations of in relation to its objective. the evidence base (Mulrow, 1987; Mignini and Khan, 2006). In this, we emphasise the distinction between a SR and a meta- The presence of these shortcomings seriously challenges the reader's analysis. A meta-analysis pools the results of a number of separate stud- ability to determine the credibility of a review. When there exist multi- ies in a single statistical analysis and may be a component of a SR; how- ple competing reviews, each using opaque methods, it becomes almost ever, it does not necessarily incorporate the full set of methodological impossible to judge their relative merits and therefore to base decisions features which define the SR process (e.g. a meta-analysis may or may on current best available evidence. The consequence is a proliferation of not include an assessment of the internal validity of included studies). conflicting opinions about best practice that fail to take proper account While we acknowledge that some researchers use the terms “system- of the body of research evidence. In the healthcare sciences, this was ini- atic review” and “meta-analysis” interchangeably, we believe the two tially shown by Antman and colleagues when they found that, in approaches should be disambiguated. It is also worth noting that comparison to recommendations of clinical experts, systematic aggre- many reviews employ a combination of narrative and systematic gation of data from existing clinical trials of streptokinase to treat myo- methods; there were differing opinions among workshop participants cardial infarction would have demonstrated benefit some years before as to the extent to which it is reasonable to expect all reviews to fully in- recommendations for its use became commonplace (Antman et al., corporate SR methods. 1992). More recently, cumulative meta-analyses have been shown to be more accurate in summarising current understanding of the size of 3.2. The current status of SR in environmental health, toxicology and CRA effect of a wide range of healthcare interventions than researchers plan- ning new clinical trials who have not used these methods (Clarke et al., While the use of SR methodologies is well established in healthcare 2014). to determine the effect of interventions on health outcomes or the accu- A SR is an approach to reviewing evidence which specifically sets out racy of a diagnostic test, application of SR is relatively novel in the fields to avoid these problems, by methodically attempting “to collate all em- of toxicology and environmental health. Workshop participants heard pirical evidence that fits pre-specified eligibility criteria in order to an- how methods for SR of medical interventions have in the United swer a specific research question,” using “explicit, systematic methods States been adapted in both academic and federal contexts to the gath- that are selected with a view to minimising bias” (Higgins and Green, ering and appraising of evidence for the effects of chemical exposures 2011). on human health: researchers at the University of California have devel- In detail, this amounts to the pre-specification of the objective and oped the Navigation Guide (Woodruff and Sutton, 2014), and the US Of- methods of the SR in a written protocol, in which the aim of conducting fice of Health Assessment and Translation (OHAT) at the US National the review is clearly stated as a structured question (for a SR of the ef- Toxicology Program has developed the OHAT Framework for systemat- fects of an intervention or exposure, this can establish a testable hypoth- ically reviewing environmental health research for hazard identification esis or quantitative parameter that is to be estimated), along with the (Rooney et al., 2014). articulation of appropriate methods. The methods specified should in- The two approaches adapt the key elements of SR methods to ques- clude the techniques for identifying literature of potential relevance to tions in environmental health (which is directly relevant to the CRA the research question, the criteria for inclusion of the studies of actual process but does not include assessment of dose–response). Features relevance to the research question, how the internal validity of the that the two approaches have in common include: conducting a SR ac- cording to a pre-specified protocol; the development of a specific research question and use of “PECO” statements (see Box 2)in “Internal validity” is a term used in Cochrane Collaboration guidance on conduct of SRs systematising review objectives and the methods that will be used to specifically intended to supersede the use of terms such as “methodological quality” or answer that question; an approach to appraising the internal validity their equivalents, which are considered ambiguous (Higgins and Green, 2011). The inter- nal validity of a piece of research is appraised in a “risk of bias” assessment. The target of of included studies adapted from the risk of bias appraisal tool devel- the risk of bias assessment is the likelihood, magnitude and direction of systematic error oped by the Cochrane Collaboration (Higgins et al., 2011); an adaptation in the size of an observed effect, as caused by flaws in the design, conduct, analysis and of the GRADE methodology (Atkins et al., 2004) for describing the cer- reporting of a study. Throughout this document, we follow Cochrane Collaboration con- tainty or strength of a body of evidence, incorporating risk of bias ele- ventions in using “internal validity” as a technical term in place of “methodological quality”. ments with other criteria such as for the assessment of relevance or Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 6 P. Whaley et al. / Environment International xxx (2015) xxx–xxx Box 2. The use of PECO statements in the SR process. external validity; and a methodology for combining the results of assessment, concentrations of a chemical in the environment and human and animal research into a statement of confidence about the biota, and the derivation of a No Observed Adverse Effect Level hazard which a chemical poses to health. (NOAEL) or Benchmark Dose Lower 95% confidence limit (BMDL). Other tools are being developed to contribute to the systematic as- European Food Safety Authority (2015c) explores these issues in more sessment of in vivo and ecotoxicity studies which have not been directly detail. derived from Cochrane Collaboration methods. Presented at the Work- Depending on scope, the resources (time and cost) to undertake an shop was SciRAP (Science in Risk Assessment and Policy), a system de- SR can be considerable. Currently there is a lack of empirical evidence veloped to improve the consistency with which the relevance and relating to the resource-effectiveness of SR approaches in CRA and there was a difference of opinion among workshop participants as to reliability of studies are appraised in the context of conducting a chem- ical risk assessment for regulatory purposes. It is also intended to reduce whether the effort required for conducting a SR tends to be under- or the risk of selection bias in the risk assessment process by providing a overestimated. It was suggested that, where effort is likely to be sub- mechanism for including non-standardised study methods yielding po- stantial, efficient use of resources may be achieved by focusing on tentially valuable data (Beronius et al., 2014; SciRAP, 2014). high-value questions developed through initial scoping exercises. For There are a number of other initiatives promoting and developing example, a low-dose adverse effect may be evident in animal models the use of SR methodologies in environmental and chemical risk assess- and supported to some extent by human epidemiology and hence a ment. Participants heard about how the European Food Safety Authority question may be formulated around this initial evidence; there may be is integrating SR methods into its assessments of food and feed safety little point, however, in pursuing a question related to non- (European Food Safety Authority, 2015b, 2015c), and about the UK carcinogenic toxicity in wildlife if a substantial part of the literature Joint Water Evidence Group methods for rapid and systematic assess- points towards that substance being a potential human carcinogen. ments of evidence (Collins et al., 2014). Other coordinated initiatives in- There is also growing interest in rapid reviews, when full SR methods clude the Evidence-Based Toxicology Collaboration (Hoffmann and are considered overly onerous (Collins et al., 2014; Schünemann and Moja, 2015). Hartung, 2006); the Collaboration for Environmental Evidence (Bilotta et al., 2014a; Land et al., 2015); and the Systematic Review Centre for The priorities for expediting the adaptation of SR methods to CRA Laboratory Animal Experimentation (SYRCLE). identified at the Workshop are as follows: 1. The development of a number of prototype CRA-focused SRs to ex- 3.3. Overcoming the challenges in implementing SR methods in CRA plore how readily SR procedures can be integrated into the CRA pro- cess, to: Risk assessment for a chemical or group of chemicals is a multi- a. identify additional methodological challenges in adapting SR faceted process that normally requires consideration of multiple end- methods to the CRA context and develop techniques to address points in relation to a variety of exposure scenarios, integrating evi- them; dence from epidemiological studies, bioassays in animals, mechanistic b. acquire practical experience in managing resources when studies and studies on the distribution and determinants of exposure conducting SRs in CRA, including the conduct of scoping exercises by different pathways and routes. In addition to resolving methodolog- for identifying high-value review questions, the further develop- ical issues relating to underdeveloped methods (e.g. how SR methods ment and/or application of novel “rapid evidence review” methods can be used as part of dose–response assessment or how they can be ap- (UK Civil Service, 2015), and how SR methods can be integrated plied to exposure assessment), it is important to consider how SR into existing regulatory structures such as REACH (see Box 3) should fit into the CRA process. One challenge going forward is to ex- (European Chemicals Agency (2/26/2015)). plore the circumstances in which applying more rigorous SR methods to assess scientific evidence would be warranted, which would require 2. Technical development of SR methodologies for CRA purposes, in insight into the practicality and cost-effectiveness of applying such particular the further advancement of techniques for appraising methods in those situations. and synthesising mechanistic, toxicological and human epidemio- In principle, it should be possible to conduct SRs in any aspect of a logical studies, to include: CRA. Given the success in employing SR methods to support evidence- a. refining tools for more consistent and scientifically robust ap- based practice in healthcare, it is intuitive that SRs could address specific praisal of the internal validity of individual studies included in a questions arising within toxicology, human epidemiology and environ- CRA and the implications for interpretation of their findings; see mental health (e.g. hazard assessment within a CRA) and this view ap- e.g. Bilotta et al. (2014b). This might include further development pears to be gaining momentum within the environmental health and validation of tools such as the SYRCLE methodology for literature. The SR method may also lend itself to answering questions assessing the internal validity of animal studies (Hooijmans et al., concerning e.g. the accuracy of the reported physical-chemical proper- 2014); for SR of observational studies see e.g. Sterne et al. (2014), ties of a substance, doses predicted by quantitative exposure Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 P. Whaley et al. / Environment International xxx (2015) xxx–xxx 7 of SR methods to CRA as a complement to current courses which largely cover SR methods in healthcare. 4. Conclusions While systematic review methods have proven highly influential in healthcare, they have yet to make widespread impact on the process of chemical risk assessment. While there is much promise in the concept of adapting SR methods to CRA to give definitive answers to specified research questions, or to enable identification of the reasons for failure to resolve debate, a number of challenges to implementing SR methods in CRA have been identified. These include particular concerns about ap- proaches to assessing bias and confounding in observational studies, the effort involved in conducting SRs, and the subsequent benefits of conforming to SR standards. Recent experience from both regulatory agencies and academics already yields some clear recommendations which would expedite the wider implementation of SR methods in CRA, potentially increasing the efficiency, transparency and scientific robustness of the CRA process. Disclaimer The views expressed in this manuscript are those of the authors and do not necessarily represent the views or policies of their employers or otherwise affiliated organisations. EA is employed by the European Food Safety Authority (EFSA); however, the present article is published under her sole responsibility and may not be considered as an EFSA scientific output. Box 3. The potential utility of SR methods in application to REACH registrations. Acknowledgements Funding for the workshop was provided through the Economic & So- cial Science Research Council grant “Radical Futures in Social Sciences” the methods employed in the NTP/OHAT and Navigation Guide (Lancaster University) and Lancaster Environment Centre. CH, PW, AR protocols, and the applicability of other assessment methods are grateful to Lancaster University's Faculty of Science & Technology such as SciRAP (Beronius et al., 2014); “Distinguished Visitors” funding programme. The Royal Society of b. the development of tools for the hazard characterisation and expo- Chemistry is acknowledged for generously providing a meeting room, sure assessment components of the CRA process; refreshments and facilitating the workshop proceedings. The PhD stu- c. the further development of software akin to the Cochrane dentship of PW is partly funded through Lancaster Environment Centre. Collaboration's Review Manager (Nordic Cochrane Centre, 2014) The contribution of non-author workshop participants to the develop- and the Systematic Review Data Repository (Ip et al., 2012), and ment of the manuscript is also greatly appreciated. toolssuchasDRAGON(ICF International, 2015) and the Health As- sessment Workspace Collaborative (Rusyn and Shapiro, 2013)to References support extraction, analysis and sharing of data from studies in- cluded in reviews; Aiassa, E., Higgins, J.P.T., Frampton, G.K., Greiner, M., Afonso, A., Amzal, B., et al., 2015. Ap- plicability and feasibility of systematic review for performing evidence-based risk as- sessment in food and feed safety. Crit. Rev. Food Sci. Nutr. 55 (7), 1026–1034. http:// dx.doi.org/10.1080/10408398.2013.769933. 3. The development an empirical evidence base for the different types Antman, E.M., Lau, J., Kupelnick, B., Mosteller, F., Chalmers, T.C., 1992. A comparison of re- of bias that operate in the CRA domain, including their direction sults of meta-analyses of randomized control trials and recommendations of clinical and potential magnitude, and the extent to which any methods experts. Treatments for myocardial infarction. JAMA 268 (2), 240–248. Atkins, D., Best, D., Briss, P.A., Eccles, M., Falck-Ytter, Y., Flottorp, S., et al., 2004. Grading being adopted to address them are appropriate and effective. quality of evidence and strength of recommendations. BMJ (Clinical research Ed.) 4. The development of a recognised “gold standard” for SRs in toxicol- 328 (7454), 1490. http://dx.doi.org/10.1136/bmj.328.7454.1490. ogy and risk assessment equivalent to the Cochrane Collaboration Beronius, A., Molander, L., Rudén, C., Hanberg, A., 2014. Facilitating the use of non- standard in vivo studies in health risk assessment of chemicals: a proposal to improve in evidence-based medicine, to address the growing number of pur- evaluation criteria and reporting. Journal of Applied Toxicology: JAT 34 (6), 607–617. ported SRs of unclear validity which are increasingly prevalent in the http://dx.doi.org/10.1002/jat.2991. environmental health literature. Bilotta, G.S., Milner, A.M., Boyd, I., 2014a. On the use of systematic reviews to inform en- vironmental policies. Environ. Sci. Pol. 42, 67–77. http://dx.doi.org/10.1016/j.envsci. 5. The creation of a climate of constructive discussion that fosters ad- 2014.05.010. vancement of methods whereby chemical risk practitioners, indus- Bilotta, G.S., Milner, A.M., Boyd, I.L., 2014b. Quality assessment tools for evidence try, competent authorities, academic researchers and policy makers from environmental science. Environ. Evid. 3(1),14. http://dx.doi.org/10.1186/ can research, discuss and evaluate SR methods and the potential ad- 2047-2382-3-14. Birnbaum, L.S., Thayer, K.A., Bucher, J.R., Wolfe, M.S., 2013. Implementing systematic re- vantages they can bring. view at the National Toxicology Program: status and next steps. Environ. Health 6. The establishment of a network of scientists and CRA practitioners to Perspect. 121 (4), A108–A109. http://dx.doi.org/10.1289/ehp.1306711. pursue research into and discussion of SR methodologies and facili- Campbell Collaboration, 2015. The Campbell Collaboration. Available online at http:// www.campbellcollaboration.org/, accessed 6/13/2015. tate their implementation. Caveman, 2000. The invited review — or, my field, from my standpoint, written by me 7. The implementation of training programmes for risk assessment using only my data and my ideas, and citing only my publications. J. Cell Sci. 113 practitioners and stakeholders, focusing specifically on application (Pt 18), 3125–3126. Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 8 P. Whaley et al. / Environment International xxx (2015) xxx–xxx Chalmers, I., Glasziou, P., 2009. Avoidable waste in the production and reporting of re- Lau, J., Rothstein, H.R., Stewart, G.B., 2013. History & progress of meta-analysis. In: search evidence. Lancet 374 (9683), 86–89. http://dx.doi.org/10.1016/S0140-6736 Koricheva, J., Gurevitch, J., Mengersen, K. (Eds.), Handbook of Meta-analysis in Ecol- (09)60329-9. ogy and Evolution. Princeton University Press, Princeton Chapter 25. Chalmers, I., Hedges, L.V., Cooper, H., 2002. A brief history of research synthesis. Evalua- Macleod, M.R., Ebrahim, S., Roberts, I., 2005. Surveying the literature from animal exper- tion & the Health Professions 25 (1), 12–37. http://dx.doi.org/10.1177/ iments: systematic review and meta-analysis are important contributions. BMJ 331 0163278702025001003. (7508), 110. http://dx.doi.org/10.1136/bmj.331.7508.110-b. Clarke, M., Brice, A., Chalmers, I., 2014. Accumulating research: a systematic account of Mignini, L.E., Khan, K.S., 2006. Methodological quality of systematic reviews of animal how cumulative meta-analyses would have provided knowledge, improved health, studies: a survey of reviews of basic research. BMC Med. Res. Methodol. 6, 10. reduced harm and saved resources. PLoS ONE 9 (7), e102670. http://dx.doi.org/10. http://dx.doi.org/10.1186/1471-2288-6-10. 1371/journal.pone.0102670. Mulrow, C.D., 1987. The medical review article: state of the science. Ann. Intern. Med. 106 Collaboration for Environmental Evidence (2015): The Collaboration for Environmental Evi- (3), 485–488. dence. Available online at http://www.environmentalevidence.org/, accessed 6/13/2015. National Food Institute, Denmark (2015): Evaluation of EFSA's new scientificopinion on Collins, A., Miller, J., Coughlin, D., Kirk, S., 2014. The Production of Quick Scoping Reviews bisphenol A. Søborg, Denmark (REG-no. DK 30 06 09 46). Available online at http:// and Rapid Evidence Assessments: A How to Guide (Beta Version 2). Joint Water Ev- www.food.dtu.dk/english/~/media/Institutter/Foedevareinstituttet/Publikationer/ idence Group. Available online at https://sbri.innovateuk.org/documents/3058188/ Pub-2015/Evaluation_BisphenolA.ashx?la=da. 3918930/The+Production+of+QSRs+and+REAs-+A+How+to+guide.pdf/ Nordic Cochrane Centre (2014): Review Manager (RevMan). Version 5.3: Cochrane Col- 45975020-be7d-4788-b74b-f3b6ed32c73a. laboration. Available online at http://tech.cochrane.org/revman, accessed 6/18/2015. European Chemicals Agency (2/26/2015), Evaluation under REACH Progress Report 2014. Plastics Europe (1/15/2015): French ban on the use of Bisphenol A (BPA) in food contact: Available online at: http://echa.europa.eu/documents/10162/13628/evaluation_ In conflict with European law and risk assessment — severe distortion of the market — report_2014_en.pdf. Accessed 4/11/2015. no safety benefit for consumers. Jasmin Bird. Available online at http://www. European Commission (1/28/2011): Directive 2011/8/EU of 28 January 2011 amending bisphenol-a-europe.org/uploads/Modules/Mediaroom/stm_re_french-bpa-ban- Directive 2002/72/EC as regards the restriction of use of Bisphenol A in plastic infant being-enforced-01-01-2015.pdf, accessed 6/15/2015. feeding bottles, Directive 2011/8/EU. In : Official Journal of the European Union. Rooney, A.A., Boyles, A.L., Wolfe, M.S., Bucher, J.R., Thayer, K.A., 2014. Systematic review Available online at http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L: and evidence integration for literature-based environmental health science assess- 2011:026:0011:0014:EN:PDF, accessed 6/15/2015. ments. Environ. Health Perspect. 122 (7), 711–718. http://dx.doi.org/10.1289/ehp. European Commission (5/31/2011): Bisphenol A: EU ban on baby bottles to enter into 1307972. force tomorrow. Brussels. Available online at http://europa.eu/rapid/press-release_ Rusyn, I.; Shapiro, A. (2013): Health Assessment Workspace Collaborative (HAWC). Ver- IP-11-664_en.htm, accessed 2/17/2015. sion Solid Hammer: UNC-CH Software. Available online at https://hawcproject.org/, European Food Safety Authority (2010): Application of systematic review methodology accessed 8/4/2015. to food and feed safety assessments to support decision making. EFSA Journal Salman, R.A.-S., Beller, E., Kagan, J., Hemminki, E., Phillips, R.S., Savulescu, J., et al., 2014. 2010; 8(6):1637. DOI: http://dx.doi.org/10.2903/j.efsa.2010.1637. Increasing value and reducing waste in biomedical research regulation and manage- European Food Safety Authority (2015a): No consumer health risk from bisphenol A ex- ment. Lancet 383 (9912), 176–185. http://dx.doi.org/10.1016/S0140-6736(13) posure. Press Release 21 Jan 2015. Parma. Available online at http://www.efsa. 62297-7. europa.eu/en/press/news/150121.htm, accessed 2/18/2015. Schünemann, H.J., Moja, L., 2015. Reviews: Rapid! Rapid! Rapid! … and systematic. Sys- European Food Safety Authority (2015b): Outcome of the targeted consultation of the tematic Reviews 4, 4. http://dx.doi.org/10.1186/2046-4053-4-4. EFSA Journal editorial on increasing openness, robustness and transparency of scien- SciRAP (2014): Science in risk assessment and policy. Department of Applied Environ- tific assessments. Available online at http://www.efsa.europa.eu/en/supporting/pub/ mental Science at Stockholm University; Institute of Environmental Medicine at 785e.htm, accessed 8/4/2015. Karolinska Institutet in Stockholm; MistraPharma. Available online at http://www. European Food Safety Authority, 2015c. Principles and process for dealing with data and scirap.org/, accessed 3/11/2015. evidence in scientific assessments. EFSA Journal 13 (5), 4121. http://dx.doi.org/10. Sena, E.S., Currie, G.L., McCann, S.K., Macleod, M.R., Howells, D.W., 2014. Systematic re- 2903/j.efsa.2015.4121. views and meta-analysis of preclinical studies: why perform them and how to ap- France (12/24/2012): LOI no. 2012–1442 du 24 décembre 2012 visant à la suspension de praise them critically. J. Cereb. Blood Flow Metab. 34 (5), 737–742. http://dx.doi. la fabrication, de l'importation, de l'exportation et de la mise sur le marché de tout org/10.1038/jcbfm.2014.28. conditionnement à vocation alimentaire contenant du bisphénol A. Legifrance.gouv. Silbergeld, E., Scherer, R.W., 2013. Evidence-based toxicology: strait is the gate, but the fr. Available online at http://legifrance.gouv.fr/affichTexte.do;jsessionid= road is worth taking. ALTEX 30 (1), 67–73. F6553AACC19D178279D8DF154EAC8558.tpdila17v_1?cidTexte= Sterne, J.A.C.; Higgins, J.P.T.; Reeves, B.C. (2014): A Cochrane Risk of Bias Assessment Tool for JORFTEXT000026830015, accessed 6/15/2015. Non-Randomized Studies of Interventions (ACROBAT-NRSI). The Cochrane Collabora- French Agency for Food, Environmental and Occupational Health & Safety (2013): tion. Available online at https://sites.google.com/site/riskofbiastool/, accessed 9/29/2014. Bisphenol A: ANSES demonstrates potential health risks and confirms the need to re- Stewart, G., 2010. Meta-analysis in applied ecology. Biol. Lett. 6 (1), 78–81. http://dx.doi. duce exposure. Available online at https://www.anses.fr/en/content/bisphenol- org/10.1098/rsbl.2009.0546. anses-demonstrates-potential-health-risks-and-confirms-need-reduce-exposure. Stewart, G.B., Schmid, C.H., 2015. Lessons from meta-analysis in ecology and evolution: French Agency for Food, Environmental and Occupational Health & Safety (4/7/2014): the need for trans-disciplinary evidence synthesis methodologies. Research Synthesis Bisphenol A: ANSES publishes its comments in response to the EFSA draft opinion for Methods 6 (2), 109–110. http://dx.doi.org/10.1002/jrsm.1152. consultation. Available online at https://www.anses.fr/en/content/bisphenol-anses- Tošenovský, E. (2014): Question for written answer to the Commission, Rule 130. publishes-its-comments-response-efsa-draft-opinion-consultation, accessed 2/18/2015. European Parliament, Parliamentary questions, P-008546/2014, Subject: possible Higgins, J P. T.; Green, S. (Eds.) (2011): Cochrane handbook for systematic reviews of in- negative impact on the internal market of measures concerning BPA adopted by terventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Avail- the French authorities. 30 October 2014. Available online at http://www.europarl. able online at http://handbook.cochrane.org/, accessed 2/18/2015. europa.eu/sides/getDoc.do?type=WQ&reference=P-2014-008546&language=EN, Higgins, J.P.T., Altman, D.G., Gøtzsche, P.C., Jüni, P., Moher, D., Oxman, A.D., et al., 2011. The accessed 6/15/2015. Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 343, Tošenovský, E. (2015): Question for written answer to the Commission, Rule 130. d5928. http://dx.doi.org/10.1136/bmj.d5928. European Parliament, Parliamentary questions, E-004315-15, Subject: measures Hoffmann, S., Hartung, T., 2006. Toward an evidence-based toxicology. Hum. Exp. Toxicol. concerning Bisphenol A. 17 March 2015. Available online at http://www.europarl. 25 (9), 497–513. http://dx.doi.org/10.1191/0960327106het648oa. europa.eu/sides/getDoc.do?pubRef=-//EP//TEXT+WQ+E-2015-004315+0+DOC Hooijmans, C.R.,Rovers, M.,de Vries,R.B., Leenaars, M., Ritskes-Hoitinga, M., 2012. An initia- +XML+V0//EN&language=en, accessed 6/15/2015. tive to facilitate well-informed decision-making in laboratory animal research: report of UK Civil Service (2015): What is a rapid evidence assessment? Available online at http:// the First International Symposium on Systematic Reviews in Laboratory Animal Science. www.civilservice.gov.uk/networks/gsr/resources-and-guidance/rapid-evidence- Lab. Anim. 46 (4), 356–357. http://dx.doi.org/10.1258/la.2012.012052. assessment/what-is, accessed 6/13/2015. Hooijmans, C.R., Rovers, M.M., de Vries, R.B.M., Leenaars, M., Ritskes-Hoitinga, M., US Environmental Protection Agency (2013): Process for developing IRIS health assess- Langendam, M.W., 2014. SYRCLE's risk of bias tool for animal studies. BMC Med. ments. Available online at http://www.epa.gov/IRIS/process.htm,accessed 6/16/ Res. Methodol. 14, 43. http://dx.doi.org/10.1186/1471-2288-14-43. 2015. ICF International (2015): DRAGON: an online tool for systematic review. Available online US Food and Drug Administration (2014): Bisphenol A (BPA): use in food contact appli- at http://www.icfi.com/insights/products-and-tools/dragon-online-tool-systematic- cation. Update on Bisphenol A (BPA) for Use in Food Contact Applications. Available review, accessed 8/4/2015. online at http://www.fda.gov/NewsEvents/PublicHealthFocus/ucm064437.htm. International Agency for Research on Cancer (2006): Preamble to the IARC Monographs. US Institute of Education Sciences (2015): What works clearinghouse. Available online at Lyon, France. Available online at http://monographs.iarc.fr/ENG/Preamble/index.php, http://ies.ed.gov/ncee/wwc/default.aspx, accessed 6/13/2015. accessed 9/10/2015. US National Research Council, 2014a. A Framework to Guide Selection of Chemical Alter- Ip, S., Hadar, N., Keefe, S., Parkin, C., Iovin, R., Balk, E.M., Lau, J., 2012. A Web-based archive natives. The National Academies Press, Washington, D.C. of systematic review data. Systematic Reviews 1, 15. http://dx.doi.org/10.1186/2046- US National Research Council, 2014b. Review of EPA's Integrated Risk Information System 4053-1-15. (IRIS) Process. The National Academies Press, Washington, D.C. Land, M., de Wit, C.A., Cousins, I.T., Herzke, D., Johansson, J., Martin, J.W., 2015. What is the US National Toxicology Panel (2015): Handbook for Conducting a Literature-Based Health effect of phasing out long-chain per- and polyfluoroalkyl substances on the concentra- Assessment Using OHAT Approach for Systematic Review and Evidence Integration. tions of perfluoroalkyl acids and their precursors in the environment? A systematic re- Available online at http://ntp.niehs.nih.gov/ntp/ohat/pubs/handbookjan2015_508. view protocol. Environ. Evid. 4(1), 3. http://dx.doi.org/10.1186/2047-2382-4-3. pdf, accessed 1/13/2015. Lau, J., Ioannidis, J.P.A., Schmid, C.H., 1998. Summing up evidence. One answer is not al- van Luijk, J., Bakker, B., Rovers, M.M., Ritskes-Hoitinga, M., de Vries, R.B.M., Leenaars, M., ways enough. Lancet 351 (9096), 123–127. http://dx.doi.org/10.1016/S0140-6736 2014. Systematic reviews of animal studies; missing link in translational research? (97)08468-7. PLoS ONE 9 (3), e89981. http://dx.doi.org/10.1371/journal.pone.0089981. Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002 P. Whaley et al. / Environment International xxx (2015) xxx–xxx 9 Vandenberg, L.N., Ehrlich, S., Belcher, S.M., Ben-Jonathan, N., Dolinoy, D.C., Hugo, E.R., et better health outcomes. Environ. Health Perspect. 122 (10), 1007–1014. http://dx. al., 2014. Low dose effects of bisphenol A. Endocrine Disruptors 1 (1), e26490. doi.org/10.1289/ehp.1307175. http://dx.doi.org/10.4161/endo.26490. Zoeller, R.T., Bergman, Å., Becher, G., Bjerregaard, P., Bornman, R., Brandt, I., et al., 2015. A Woodruff, T.J., Sutton, P., 2014. The Navigation Guide systematic review methodology: a path forward in the debate over health impacts of endocrine disrupting chemicals. rigorous and transparent method for translating environmental health science into Environ. Heal. 14, 118. http://dx.doi.org/10.1186/1476-069X-13-118. Please cite this article as: Whaley, P., et al., Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environ Int (2015), http://dx.doi.org/10.1016/j.envint.2015.11.002

Journal

Environment International – Unpaywall

Published: Jul 1, 2016

There are no references for this article.