Concise Research Reports
What Constitutes an Independent Statistical Analysis?
Ann Abraham, BS
, Alexandra Woodbridge, BS, BA
, Erin Madden, MPH
Salomeh Keyhani, MD, MPH
, and Deborah Korenstein, MD
San Francisco VA Medical Center, San Francisco, CA, USA;
Oregon Health and Science University, Portland, OR, USA;
Tulane University School of
Medicine, New Orleans, LA, USA;
University of California, San Francisco, CA, USA;
Memorial Sloan Kettering Cancer Center, New York, NY, USA.
KEY WORDS: independent statistical analysis; evidence-based medicine;
J Gen Intern Med 33(6):786–8
© Society of General Internal Medicine 2018
Potential bias in clinical trials related to relationships with the
pharmaceutical industry is a longstanding concern.
2005 and 2013, JAMA journals required industry-sponsored
studies to conduct independent statistical analysis (ISA), de-
fined as data analysis by an Bindependent statistician at an
academic institution^ using the raw data set.
journals currently require ISA, the term may be used to denote
impartiality and robustness in data analysis.
meaning, frequency of use, and association with study char-
acteristics are not clear. Our study’s purpose was to investigate
the prevalence and characteristics of ISA in published RCTs
focused on drug efficacy and their adherence to JAMA’s
We searched MEDLINE and randomly selected 646 drug effi-
cacy RCTs from 2013, as described previously;
inclusion criteria. Two of four reviewers (AA, RA, AW, SS)
independently abstracted data regarding trial characteristics,
clinical area, results, funding source, investigator/manufacturer
financial ties, and description of ISA or independent statistician.
Among papers reporting ISA, we abstracted in duplicate infor-
mation concerning the analysis the sponsor’s relationship to
data and analyses, and statistician(s) identity. Disagreements
were resolved by consensus. When ISA was described, we
determined conformity with its definitional components (aca-
demic statistician affiliation and use of the full dataset) and the
relationship between ISA and study characteristics and out-
come. We used the Mann–Whitney test for continuous variables
and Chi-squared for categorical variables (SAS, V9).
Statistical Analysis Characteristics
Among the 190 trials, 17 (8.9%) reported ISA; the majority
(15, 88%) were industry-funded and published in high impact
journals (IF > 10) (12, 71%) (Table 1). Most identified the
independent statistician(s) by name (11, 65%). Roles of inde-
pendent statisticians varied; they led the analysis in eight trials
(47%), validated the sponsors analysis in four (24%), provided
statistical assistance in three (18%), and had an unspecified
role in two (12%). ISA adhered to both components of the
definition in seven trials (41%); independent analysts had
academic affiliation in 13 trials (76%) and full dataset access
in 11 (65%).
Relationship to Study Characteristics
ISAwas not associated with industry funding (p value = 0.07),
positive study outcome (p value = 0.31), or financial ties to the
manufacturer (p value = 0.42). ISA was strongly associated
with sample size (p value < 0.0001) and clinical area (p value
< 0.001), notably cardiology. ISAwas not associated with trial
registration, analysis type, phase, comparator, outcome mea-
sure, or first author country (Table 1).
We found that drug efficacy RCTs rarely self-reported ISA,
though the term was used more commonly in large, industry-
funded studies published in high impact journals. The mean-
ing of ISA varied among trials with some statisticians control-
ling the analysis and others serving as collaborators or
In the past, JAMA clearly defined ISA and required it
to ensure integrity and minimize bias,
but this require-
ment resulted in fewer manuscript submissions by indus-
try and was dropped.
Regardless, the term remains in
use. Our findings demonstrate ambiguity around its mean-
ing, possibly resulting in an unwarranted implication of
rigor and integrity. Given this ambiguity, readers of the
literature should not assume that ISA represents method-
ological rigor. Instead, readers concerned about the integ-
rity of data analysis should note details of the identity,
role, and affiliation of authors or statisticians performing
Ann Abraham, Rosa Ahn and Alexandra Woodbridge contributed equally
to this work.
Published online March 16, 2018