Editorial
The challenge of objective assessment of surgical skill
Ara Darzi, Vivek Datta, Sean Mackay*
Academic Surgical Unit, Imperial College School of Medicine, 10th Floor, QEQM Building, St Mary’s Hospital, South Wharf Rd, London W2 1 NY UK
Technical performance in surgery has come under in-
creased scrutiny in recent years, not least due to several
highly publicized cases where it has been suggested that
poor outcomes were the result of inadequate technical skill
[1–3]. In the Bristol case, which involved a pediatric
cardiac surgery unit, the concerns voiced by a member
of medical staff have now resulted in disciplinary action
against two surgeons by the General Medical Council,
and a judicial inquiry, which is yet to hand down its report
(www.bristol-inquiry.org.uk). The resulting debate, both
public and within the profession, has focused on the need
for objective and independent assessment of surgical
skill, and a significant research interest has developed in this
area.
Methods of assessment
There are now several methods of providing an objective
assessment of performance available [4–6]. Some of these
are well established whereas others are research tools in the
process of evaluation. In general, these methods involve a
standardized set of tasks. Most assessment is carried out in
the laboratory setting, although there are some techniques
that are amenable for use in the operating room. Our aim in
this article is not to review these objective methods of
assessment, which are examined elsewhere, but rather to
discuss the issues that arise in assessing the individual;
however, it is pertinent to present a brief summary of each
at this stage.
Objective Structural Assessment of Technical Skill
The Objective Structured Assessment of Technical Skill
(OSATS) was first used by Martin in her 1997 paper, writ-
ten while working in Toronto with Reznick. The basic
principle is to use carefully designed and administered
checklists to reduce the subjectivity of the observer’s expe-
rience. There is now a significant body of work that dem-
onstrates the reliability and effectiveness of the OSATS in
assessing surgical performance. In our unit, we have found
it more useful for simpler tasks, and hence for more junior
trainees—the use of OSATS in the setting of “error detec-
tion” is a challenging concept that could conceivably extend
the utility of this assessment modality.
Imperial College Surgical Assessment Device
The Imperial College Surgical Assessment Device uti-
lizes commercially available electromagnetic tracking
equipment (Isotrak II, Polhemus Inc.) and bespoke soft-
ware, developed within our department, to track a surgeon’s
hands as he or she performs a standardized surgical task.
The system generates simple Cartesian coordinates for each
tracker at 20 Hz, and the software then generates data for
total path length through the air, total number of move-
ments, and the time taken. This device has proved useful in
assessing surgeons and trainees as they perform standard-
ized tasks [7].
Competence Day
This concept is essentially that of an Objective Struc-
tured Clinical Examination (OSCE) in technical skills. By
reference to a simple model of competence, the chosen tasks
are designed to complement each other, and hence assess all
aspects of technical ability. We have developed an OSCE
for basic surgical trainees (equivalent to postgraduate year
1/2 residents). The examination consists of six tasks: knowl-
edge of instruments, sutures, and surgical equipment (a
simple answer sheet); knot tying (ICSAD); skin-pad sutur-
ing (ICSAD); closure of an enterotomy in synthetic small
bowel (video-based OSATS); excision of a skin lesion
(video-based OSATS); and laparoscopic skills on the min-
imally invasive surgical trainer (MIST VR, Virtual Pres-
ence, London, United Kingdom).
* Corresponding author. Tel.: ϩ44-207-886-1310; fax: ϩ44-207-413-
0470
E-mail address: v.datta@ic.ac.uk (S. Mackay).
The American Journal of Surgery 181 (2001) 484–486
0002-9610/01/$ – see front matter © 2001 Excerpta Medica, Inc. All rights reserved.
PII: PII S0002-9610(01)00624-9