Validating Measures of Third Year Medical Students’ Use
of Interpreters by Standardized Patients and Faculty Observers
Desiree Lie, MD, MSEd
, John Boker, PhD
, Sylvia Bereknyei, MS
, Susan Ahearn, RN
Charlotte Fesko, BA
, and Patricia Lenahan, LCSW, MFT
University of California, Irvine School of Medicine, Irvine, CA, USA;
Stanford University School of Medicine, Stanford, CA, USA;
Family Medicine, Orange, CA 9286, USA.
BACKGROUND: Increasing prevalence of limited En-
glish proficiency patient encounters demands effective
use of interpreters. Validated measures for this skill are
OBJECTIVE: We describe the process of creating and
validating two new measures for rating student skills
for interpreter use.
SETTING: Encounters using standardized patients
(SPs) and interpreters within a clinical practice exami-
nation (CPX) at one medical school.
MEASUREMENTS: Students were assessed by SPs
using the interpreter impact rating scale (IIRS) and the
physician patient interaction (PPI) scale. A subset of 23
encounters was assessed by 4 faculty raters using the
faculty observer rating scale (FORS). Internal consis-
tency reliability was assessed by Cronbach’s coefficient
alpha (α). Interrater reliability of the FORS was exam-
ined by the intraclass correlation coefficient (ICC). The
FORS and IIRS were compared and each was correlated
with the PPI.
RESULTS: Cronbach’s α was 0.90 for the 7-item IIRS
and 0.88 for the 11-item FORS. ICC among 4 faculty
observers had a mean of 0.61 and median of 0.65 (0.20,
0.86). Skill measured by the IIRS did not significantly
correlate with FORS but correlated with the PPI.
CONCLUSIONS: We developed two measures with good
internal reliability for use by SPs and faculty observers.
More research is needed to clarify the reasons for the lack of
concordance between these measures and which may be
more valid for use as a summative assessment measure.
KEY WORDS: interpreter use; evaluation of skills; assessment;
J Gen Intern Med 22(Suppl 2):336–40
© Society of General Internal Medicine 2007
ecent emphasis on cultural competence curricula in
medical education has highlighted the need for training
medical students and residents in the effective use of inter-
The knowledge and application of this particular
skill set is distinctly described as a separate learning entity
(domain V) in the AAMC’s tool for assessing cultural compe-
tency training (TACCT).
Several recent guidelines have
reported areas of consistency and agreement, as well as
controversy, about the best practices for interpreter use from
both interpreter and clinician perspectives.
Despite the need to train learners to use interpreters
effectively and the availability of guidelines describing core
skills, validated measures for evaluating these skills are not
available. The challenges of designing such measures for
clinical encounters are considerable. They range from the
diverse and varying skill levels of interpreters available in
practice settings to the difficulty of isolating communication
skills necessary for using an interpreter to the shortage of
faculty fluent in the patient’s language to assess the verbal
content of non-English language encounters. Partial provider/
learner fluency in the patient’s language may hinder or help
the encounter when an interpreter is present. In addition, the
manner in which clinicians utilize interpreters,
awareness of the health literacy of their limited English
proficiency (LEP) patients are also important variables affect-
ing physician–patient communication,
length of visits,
and the quality of health care and patient satisfaction.
attempts to design assessment tools of interpreter use have
been reported in the literature, and those that have are case-
specific and not generalizable across other encounters using
In this study, we used an empirical expert-based consensus
framework to create two interpreter use skill assessment
measures for clinical encounters where an interpreter is
involved. We designed a study to test the internal reliability of
the two measures and their performance compared to a widely
used communication assessment measure in the setting of a
standardized clinical examination.
We used an iterative process to create two measures: the
interpreter impact rating scale (IIRS), for use by standardized
patients (SPs), and the faculty observer rating scale (FORS), for
use by faculty. The rationale for creating a new SP measure to
assess interpreter-mediated communication was that addi-
tional or different observable skills apply to an encounter when
an interpreter is present as a third party, which may impact
patient/SP perception of the provider and perceived effective-
ness of the information transfer.