DOI 10.1007/s11135-006-9057-z
Quality & Quantity (2008) 42:513–525 © Springer 2006
Using Clinical Trials to Benchmark Effects
Produced in Clinical Practice
TAKUYA MINAMI
1,
, RONALD C. SERLIN
2
, BRUCE E.
WAMPOLD
3
, JOHN C. KIRCHER
1
and G.S. (JEB) BROWN
4
1
Department of Educational Psychology, University of Utah, Salt Lake City, UT, USA;
2
Department of Educational Psychology, University of Wisconsin, Madison, WI, USA;
3
Department of Counseling Psychology, University of Wisconsin, Madison, WI, USA;
4
Center for Clinical Informatics, Salt Lake City, UT, USA
Abstract. This paper proposes an intuitive yet statistical advancement of the benchmarking
method (e.g., Weersing and Weisz, 2002, Journal of Consulting and Clinical Psychology 70:
299–310) that could facilitate the assessment of pre-post treatment effectiveness of psycho-
therapy and other interventions delivered in clinical settings against efficacy observed in clin-
ical trials. Primary development was in the use of the “good-enough principle” (Serlin and
Lapsley, 1985 American Psychologist 40: 73–83, 1993, In: G. Keren & C. Lewis (eds.), A
handbook for Data Analysis in Behavioral Sciences: Methodological Issues. Hillsdale, NJ:
Lawrence Erlbaum Associated, pp. 199–228), which allowed for setting a clinically relevant
margin between the benchmarks and the effect sizes observed in clinical settings so as to
avoid attaining statistical significance with clinically trivial differences. Examples are given
using clinical trials benchmarks of adult depression treatment, followed by instructions and
limitations for its use.
Key words: benchmarking, efficacy and effectiveness, meta-analysis, psychotherapy, clinical
practice
1. Introduction
Although psychotherapy efficacy (i.e., effect of psychotherapy in clinical
trials) has been well established, psychotherapy effectiveness (i.e., effect
of psychotherapy in clinical settings) has not. In the past decade, inter-
est in evaluating effectiveness has been emphasized (e.g., Seligman, 1995;
Goldfried and Wolfe, 1998; Shadish et al., 2000). However, investigations
regarding effectiveness of treatment-as-usual (TAU) in these settings have
Author for correspondence: Takuya Minami, Department of Educational Psychol-
ogy, University of Utah, 1705 East Campus Center Drive RM 342, Salt Lake City, UT,
84112–9599, USA. Tel.: (1)-801-5817191; Fax: (1)-801-5815566; E-mail: takuya.minami@
ed.utah.edu
@Phil_Robichaud
@deepthiw
@JoseServera