Vol:.(1234567890)
Cognitive Therapy and Research (2018) 42:358–368
https://doi.org/10.1007/s10608-018-9898-5
1 3
ORIGINAL ARTICLE
Observational Coding Training Methods for CBT Treatment Fidelity:
A Systematic Review
Natalie Rodriguez‑Quintana
1
· Cara C. Lewis
1,2,3
Published online: 22 February 2018
© Springer Science+Business Media, LLC, part of Springer Nature 2018
Abstract
Observational coding is the gold standard for coding treatment fidelity in Cognitive Behavioral Therapy (CBT). This method
allows researchers to objectively assess therapist behaviors to determine if CBT was delivered as intended. However, there
is no standard method or protocol that ensures reliability within and between research groups, compromising compara-
bility between studies. The aim of this study was to evaluate procedures used to train raters to quantify CBT fidelity. A
systematic review was conducted. PsycINFO and Web of Science databases were reviewed for articles up until July 2015;
the parameters of the search included CBT, depression, treatment fidelity, and coding/rating. A total of 521 articles were
screened for inclusion, which yielded 81 articles. Data was extracted across four categories (study characteristics, training
process, inter-rater reliability, and rating), and 62 codes. The results revealed variability in training approaches (e.g., 25.9%
of articles reported 10–50 h of training). Very little information was provided about the training process (i.e. only one study
specified training duration). Training materials were: manuals (21.0%), videos (18.5%), audiotapes (2.4%), readings (1.2%),
and lectures (1.2%). Almost all (90.1%) studies did not describe the trainer, and 28.4% did not describe raters. The relation
between CBT fidelity and clinical outcomes is equivocal, perhaps because intra-team and inter-team reliability is difficult
to achieve. This systematic review revealed variability in training others to rate CBT fidelity, of which some approaches
might inform the field for future use.
Keywords Systematic review · Observational coding · Training · Fidelity · CBT
Introduction
Treatment fidelity, also referred to as treatment integrity,
refers to the degree to which a treatment has been imple-
mented as intended. Fidelity is often decomposed into three
components: (1) adherence, (2) competence, and (3) differ-
entiation (Perepletchikova et al. 2007; Waltz et al. 1993;
Webb et al. 2010). Adherence refers to the level of faithful-
ness to the theory and/or method of the intervention. Compe-
tence refers to the degree of skill to which that intervention
has been implemented. Differentiation, as defined by Bor-
relli (2011), refers to the delivery of the intended interven-
tion and not other possible intervention components (i.e.,
other treatment modalities). Assessing fidelity is critical in
randomized controlled trials (RCT) to conclude that differ-
ences in outcomes are due to treatment conditions. Fidelity
assessment is also essential in the context of implementation
efforts to determine if the therapy is being delivered in com-
munity settings as intended. Assessing fidelity has also been
increasingly incorporated in process research and as a core
feature of supervision (Lewis et al. 2014).
Observational coding is the gold standard methodology
for assessing treatment fidelity. Observational methods refer
to the use of operational definitions of desired behaviors that
are measured by trained observers (Bakeman and Gottman
Electronic supplementary material The online version of this
article (https ://doi.org/10.1007/s1060 8-018-9898-5) contains
supplementary material, which is available to authorized users.
* Natalie Rodriguez-Quintana
natarodr@indiana.edu
Cara C. Lewis
lewis.cc@ghc.org
1
Department of Psychological and Brain Sciences, Indiana
University, 1101 E 10th St, Bloomington, IN 47405, USA
2
Kaiser Permanente Washington Health Research Institute,
MacColl Center for Health Care Innovation, 1730 Minor
Avenue, Suite 1600, Seattle, WA 98101, USA
3
Department of Psychiatry & Behavioral Sciences, University
of Washington, 325 Ninth Street, Seattle, WA 98104, USA
@Phil_Robichaud
@deepthiw
@JoseServera