© 2018 Macmillan Publishers Limited, part of Springer Nature. All rights reserved. © 2018 Macmillan Publishers Limited, part of Springer Nature. All rights reserved.
Department of Biology, Whitman College, Walla Walla, WA, USA.
Department of Biological Sciences, Macquarie University, North Ryde, New South
Department of Ecology and Evolutionary Biology, University of Arizona, Tucson, AZ, USA.
School of BioSciences, University of
Melbourne, Melbourne, Victoria, Australia.
History & Philosophy of Science, School of Historical & Philosophical Studies, University of Melbourne,
Melbourne, Victoria, Australia.
Department of Biology, Clark University, Worcester, MA, USA.
Department of Behavioural Ecology and Evolutionary
Genetics, Max Planck Institute for Ornithology, Seewiesen, Germany.
Department of Ecology and Evolution, Stony Brook University, Stony Brook, NY,
School of Biological Sciences, Royal Holloway University of London, Egham, Surrey, UK.
Department of Computational Landscape Ecology, UFZ
– Helmholtz Centre for Environmental Research, Leipzig, Germany.
Institute of Geoscience and Geography, Martin-Luther-University Halle-Wittenberg,
Halle (Saale), Germany.
iDiv – German Centre for Integrative Biodiversity Research Halle-Jena-Leipzig, Leipzig, Germany.
Department of Ecology and
Evolutionary Biology, University of Connecticut, Storrs, CT, USA.
Evolution & Ecology Research Centre, School of Biological, Earth and Environmental
Sciences, University of New South Wales, Randwick, New South Wales, Australia. *e-mail: firstname.lastname@example.org
wo important tasks facing peer reviewers are assessing the
soundness of study design and evaluating the reporting of
methods and results. Study soundness and reporting both
bear directly on the reliability of the inferences that can be drawn
from the papers that are ultimately published
. Other reviewing
tasks include considering the placement of the study in a broader
context, the writing and the importance of the research, but these
vary by journal and the expertise of the reviewer, and are often more
subjective. We therefore focus on only the first two reviewing tasks.
Our goal here is to explain particular components of this assessment
process that we believe are too frequently ignored by peer reviewers,
ultimately to the detriment of the scientific literature. We combine
these components in a checklist that reviewers can use to improve
transparency and reduce bias, and thus improve the reliability of
We present this checklist as a series of ten questions (summa
rized in Box 1 and Supplementary Information), each accompanied
by suggestions for how the reviewer should proceed depending on
the answer to that question. The checklist is not meant to be com
prehensive. A longer checklist to help reviewers in ecology and evo-
lutionary biology promote transparency was created as part of Tools
for Transparency in Ecology and Evolution (TTEE; https://osf.io/
y8aqx/) in an effort to help journals in ecology and evolutionary
biology adopt transparency and openness promotion guidelines
TTEE checklists, for both reviewers and authors, were designed to
cover a broad swath of transparency issues. In contrast, the short
checklist we present in this paper focuses on the subset of practices
that we think are critically in need of improvement, and on which
we think a concise checklist can achieve greatest impact. Our check
list provides reviewers with an efficient tool for promoting transpar-
ency in empirical research papers.
Why a checklist?
The use of checklists is well established among skilled practitioners
working in complex systems. Checklists make flying complicated
aircraft safer, they free architects to devote their mental energy to
creativity and they help surgeons focus on applying their skill with
out forgetting vital tasks
. Good checklists do not replace complex
thought; they facilitate it. Of course, effective peer review requires
expertise and critical thinking skills that no practical checklist can
provide. However, this does not mean that checklists cannot be used
to improve peer review, even dramatically, by calling attention to
essential elements that are often overlooked.
Checklists can be of use to peer reviewers in two primary ways
related to creating a more transparent and less biased literature: to
help reviewers check (1) mundane but important details, and (2)
both their own and the authors’ potential biases. With regard to the
first point, incomplete reporting of information hinders interpreta
tion of studies and effective synthesis, and thus scientific progress
We know from surveys of subsets of the ecology literature that
approximately half of published papers omit important information
such as sample size or variability associated with estimates
all papers omitting this information were peer reviewed, suggest
ing that reviewers either overlooked these details, or felt that it was
someone else’s job to monitor them. Whether we notice omissions
as reviewers depends on scrutiny that may vary unconsciously with
Empowering peer reviewers with a checklist to
Timothy H. Parker
*, Simon C. Griffith
, Judith L. Bronstein
, Fiona Fidler
, Susan Foster
, Wolfgang Forstmeier
, Jessica Gurevitch
, Julia Koricheva
, Ralf Seppelt
Morgan W. Tingley
and Shinichi Nakagawa
Peer review is widely considered fundamental to maintaining the rigour of science, but it often fails to ensure transparency and
reduce bias in published papers, and this systematically weakens the quality of published inferences. In part, this is because
many reviewers are unaware of important questions to ask with respect to the soundness of the design and analyses, and
the presentation of the methods and results; also some reviewers may expect others to be responsible for these tasks. We
therefore present a reviewers’ checklist of ten questions that address these critical components. Checklists are commonly
used by practitioners of other complex tasks, and we see great potential for the wider adoption of checklists for peer review,
especially to reduce bias and facilitate transparency in published papers. We expect that such checklists will be well received by
NATURE ECOLOGY & EVOLUTION | VOL 2 | JUNE 2018 | 929–935 | www.nature.com/natecolevol