TY - JOUR
AU - Darzi, Ara
AB - HypothesisObjective assessment of surgical skill has recently been shown to be possible through the use of dexterity-based and video analysis systems. The aim of this study was to synchronize these 2 modalities to produce a comprehensive surgical assessment tool.DesignThe Imperial College Surgical Assessment Device is a dexterity-based motion analysis device that has been developed in the Department of Surgical Oncology and Technology by the Surgical Computing and Imaging Research Group. Further advances to this system have been made to enable synchronized acquisition of hand kinematics and video from real procedures, and their concurrent analysis. To test the feasibility of the system, 10 laparoscopic cholecystectomies performed by 5 different surgeons on consenting patients were recorded. Analysis focused on the entire procedure and also on specific parts of the operation such as the clipping and cutting of the cystic duct and artery.ResultsDexterity analysis was performed using the objective measures of time, path length, number of movements, velocities, and trajectories. Comparative analysis of a surgeon’s dexterity was carried out on the whole procedure and by using the synchronized zoom facility in the software. Kinematic signals revealed rapid changes in velocity caused by alternating between different instruments or occurring after complications such as bleeding.ConclusionThis new motion analysis system has been shown to be an effective tool for the comprehensive assessment of operative procedures.Recent high-profile cases of medical mishaps have increased public and subsequent political pressure of the need to objectively assess the competence of all surgeons.However, this has traditionally been performed in a subjective manner according to the master-apprenticeship model. This mode of assessment is known to be biased, and recent literature has commented on the fact that surgical assessment needs to be made more objective.Furthermore, Kaufman et aland Reznick et alhave remarked on the need for objective feedback during the process of surgical training. Without feedback, learning is inadequate, as deficiencies in skills cannot be properly addressed.The lack of feedback in the acquisition of surgical skill was clearly evident from the large number of unnecessary complications that occurred during the introduction of laparoscopic surgery.Without standardized and assessed methods of training, it is difficult to assure the public of the credentials of the surgeons operating on them. To achieve this, courses were introduced to teach the basic psychomotor skills required for laparoscopic surgery. Inexperienced surgeons had to perform tasks in the skills laboratory and were assessed subjectively by expert observers.More recently, there has been an influx of tools enabling the objective assessment of psychomotor skills acquisition in both open and laparoscopic surgery. These can be broadly divided into motion analysis systems such as the Imperial College Surgical Assessment Device (ICSAD)and Advanced Dundee Endoscopic Psychomotor Tester (ADEPT),and video-based assessment tools such as Objective Structured Assessment of Technical Skill (OSATS).The ICSAD makes it possible to quantifiably assess surgical dexterity by comparing the performance of experienced and novice surgeons on a simple surgical task. An experienced surgeon would be able to complete the task by making fewer movements and with a lower path length than an inexperienced trainee.During training sessions, it would also be possible for the trainee to view a benchmark to be achieved in terms of dexterity. It is also possible to objectively assess dexterity for criteria-based progression from task performance such as targeting, transfer, and dissection in laparoscopic surgery, through to procedural performance that integrates these skills into a complete laparoscopic cholecystectomy.However, the ICSAD trace does not provide any information regarding the quality of the operation performed nor its outcome. Video-based assessment tools do enable qualitative assessment of skillin an objective manner though the process is labor intensive and must involve expert surgeons to perform the ratings. The development of technology that could integrate computerized dexterity analysis with video-based assessment would perhaps be a more efficient and comprehensive method of surgical skills assessment. It is the aim of this article to describe the development of such a tool and to confirm the feasibility of using it as a skills assessment device in the real operating theater.METHODSDEVELOPMENT OF THE ROVIMAS SOFTWAREThe term “movement” is defined as a significant velocity peak, made up of an acceleration followed by a deceleration. In ICSAD, a movement is calculated from the time-stamped Cartesian positional data derived from the electromagnetic sensors, each one attached to the dorsum of the surgeon’s hands. Previous studies have validated this conceptin combination with qualitative assessment methods.A movementwas defined as an instant acceleration from one local static point followed by a deceleration to another static point. An a priori condition stated that each movement is counted when the physical movement high-peak velocity value exceeds a predefined velocity tolerance. Regardless of wrist movement or upper limb repositioning, it is the movements of the sensors placed on the dorsal surface of the hands that are recorded. As long as the a priori condition is satisfied, the parameter movement is counted at a specific time stamp.In motion analysis of hand movement, it is essential to separate a deliberate movement from human hand tremor and measurement error, which form the background noise. Therefore, a calculation of the velocity threshold was integrated into the software. To set this tolerance, we measured the background noise by placing both trackers on the bench (not attached to hands) to record only noisy velocity values. A value of 12 mm/s was found to be the noise threshold. Moreover, the elimination of small insignificant movements due to hand tremor was achieved by attaching the receivers to hands, which were placed on the experimental bench. It was found that a value of 15 mm/s could adequately remove the background noise and hand tremor, discriminating insignificant from significant movements. In addition to the number of movements made by each hand, the system also calculated the time taken for the task, the distance traveled by each hand, and the average speed.The ICSAD system consists of 3 components. A commercially available electromagnetic tracking system (Isotrak II; Polhemus Inc, Colchester, Vt) connected to a portable computer through a standard RS-232 (serial) port, independent motion acquisition software, and custom-made analysis software used for converting the positional data of the trackers into dexterity measures. The electromagnetic tracking system (Figure 1) consists of an electromagnetic field generator and 2 sensors that are attached to the dorsum of the surgeon’s hands.Figure 1.Imperial College Surgical Assessment Device (ICSAD). The ICSAD system consists of 3 components. A commercially available electromagnetic tracking system (Isotrak II; Polhemus Inc, Colchester, Vt) connected to a portable computer through a standard RS-232 (serial) port, an independent motion acquisition software, and bespoke analysis software for converting the positional data of the trackers into dexterity measures. The Isotrak II system consists of an electromagnetic field generator and 2 sensors that were attached to the dorsum of the surgeon’s hands.The ICSAD device was integrated into a large-scale project involving the assessment of model-based tasks and real procedures carried out using robotic, conventional laparoscopic, as well as open surgery. New software, the Robotic Video and Motion Analysis Software (ROVIMAS), was developed to provide concurrent qualitative and quantitative functionalities.This is based on the same framework as the original ICSAD analysis software and enables compatibility when comparing old and new data sets. All ICSAD components (acquisition and data analysis software) were integrated into ROVIMAS, with data retrieval and analysis automatically performed by the new software. Additional visual functionality includes Cartesian coordinates and distance and velocity graphs, as well as hand direction, polar graph, and trajectories. Moreover, a new efficient graphical method called “density analysis” provides information regarding which part of the surgical scene was the focus of activity, the active scene surface, and the total task surface.The zooming functionality is of significant importance as it allows isolation of various parts of procedures to be broken down according to the particular assessment process, saving valuable time. Each time zooming occurs, the number of movements and of hands’ path length are shown and recalculated with all graphs being updated accordingly.The ROVIMAS software also includes a video frame–capturing facility (Figure 2). Motion and video data are captured in a time-synchronous manner achieving synchronization accuracy of less than 40 milliseconds.Video frames are compressed in real time using the Microsoft VKI codecs (Microsoft Corporation, Redmond, Wash) with a pixel size of 320×240. It is also possible to capture single full-sized bitmap pictures while recording a task.Figure 2.Imperial College Surgical Assessment Device (ICSAD) data and video acquisition component. On the left side information regarding the commercially available electromagnetic tracking system (Isotrak II; Polhemus Inc, Colchester, Vt) is displayed along with the tracking status. On the right the video capturing window displays the endoscopic surgical scene. Both motion and video streams are recorded using the same time stamp succeeding in perfect synchronization.The main advantage of this system is that through video synchronization it is possible to assess the operation using the ROVIMAS-integrated video player (Figure 3) and to concurrently study the corresponding motion signal trace. An application that identifies high-velocity movements and associates them with erroneous movements was developed previously. A playback line bar has been implemented on the signal trace to enable searching for a particular part of the procedure, and the dexterity graph will change in synchrony. Alternatively, one can zoom into the desired signal regions and the video will play only the required part and not the whole operation. ROVIMAS also uses the same concept of motion and video synchronization when data are acquired from the da Vinci telemanipulator system (Mountain View, Calif).Figure 3.Snapshot of the Robotic Video and Motion Analysis Software (ROVIMAS) showing the crown pattern during a laparoscopic cholecystectomy case in which each pair of 2 high-velocity peaks represents extraction-insertion of the scissors-stapler instrument. Users can browse these data using the “browsing bar” and watch the corresponding video frames from the video player. P indicates path.DATA COLLECTIONThe ROVIMAS system was used to record and analyze 10 laparoscopic cholecystectomies performed by 5 surgeons: 1 experienced surgeon (A.D.) (>100 laparoscopic cases) and 4 inexperienced surgeons (<50 cases) between October 1, 2003, and February 28, 2004. Ethics approval for this study was obtained from the local research ethics committee.Our intention was not to produce assessment results from real procedures but rather to explore the feasibility of using this system in the operating room. This was to ensure that data could be reliably recorded for the whole and parts of the procedure, that the operative environment (instruments, anesthetic equipment, diathermy, and others) would not cause signal disturbance, and, most importantly, that the electromagnetic field would be safe from a patient’s perspective. Data recording began with insertion of the endoscopic camera into the abdominal cavity and terminated once the gallbladder had been extracted from the abdomen.RESULTSTen laparoscopic cholecystectomies were recorded, and accurate data were available for all operations. Measurements illustrating the entire procedure and breakdown into cystic duct dissection/clipping and cutting from 10 recorded cholecystectomies are given in the Table. Objective data parameters (number of movements, path length, and time taken) were calculated for entire operations and for dissecting/clipping and cutting of the cystic duct. Left movement (LM) and right movement (RM) represent the left-hand and right-hand movements, respectively. Left-path length (LPL) and right-path length (RPL) characterize the left-hand and right-path hand length, respectively. Additionally, it would have been possible to perform qualitative analysis on the whole procedure, or selected parts, using a validated objective qualitative method of assessment, such as OSATS, and correlate it with the quantitative data.Table. The Number of Movements, Time, and Path Length for the Entire Task, the Dissecting and Clipping and Cutting of the Cystic DuctSurg IDCholecystectomy No.Entire TaskTime, sNo. of LMsNo. of RMsLPL, mRPL, mExpert 11826.481191178.286.72735.177983871.9862.363793.9771104879.0994.32Novice 242897.615842771128.16189.035992.4718116660.1988.81Novice 36826.6770106367.9592.137942.5732110963.0492.38Novice 481037.2731143545.0591.54Novice 593080.81228275496.29203.25101843.3865194874.78143.12Cystic DuctDissectingExpert 11118.9971588.4513.982172.21402049.1214.55375.1631075.598.48Novice 24743.921964816.7642.00582.417881.745.62Novice 36179.1872116.2714.067253.214628912.0022.24Novice 48466.525165512.4636.35Novice 59565.516543411.1624.5610578.127565519.7042.91Clipping and CuttingExpert 1153.246673.785.49225.614360.862.47333.634482.414.18Novice 2463.67520.853.64519.14240.332.46Novice 3660.644951.897.89735.917350.953.51Novice 485822581.004.68Novice 59119.735942.275.911088.732812.027.37Abbreviations: LM, left movement; LR, right movement; LPL, left-path length; RPL, right-path length.The recorded procedures presented different levels of difficulty. It was our intention to display the widest possible range of operative difficulty possible, thus demonstrating a wide variety of results. Surgeons had to deal with complications such as acute cholecystitis or a large left lobe of the liver. The combination of operative difficulty and surgical expertise demonstrates a variation in operative duration of between 735.1 and 3080.8 seconds. Notwithstanding duration as a significant factor, it cannot be used as a stand-alone criterion to determine surgical dexterity.Figure 3shows a snapshot of the dissection of the cystic duct. Data from the red signal correspond to the right hand while those from the green signal correspond to the left hand. The “crown pattern” illustrated can be recognized in all procedures, where each pair of 2 high-velocity peaks represents extraction-insertion of the scissors-stapler instrument. By zooming in only on in that part between 110 and 311.4 second, the number of movements was found to be 240 for the left hand and 380 for the right hand, indicating greater activity for the right hand (dissection part of the procedure). Figure 4shows the clipping and cutting of the cystic duct where the time spent was 13 and 17 seconds, respectively. For the right hand, the numbers of movements were 29 and 32 and the path length was 0.898 and 1.137 m.Figure 4.Velocity data (right hand) for clipping (A) and cutting (B) the cystic duct as appear in the Robotic Video and Motion Analysis Software (ROVIMAS). Screenshot shows higher hand activity during cutting (denser velocity peaks) compared with clipping that requires more precise movements (wider peaks). P indicates path.Listed in the Tableis analysis of data from cases 1 and 6 in which the 2 surgeons had similar operative times, though they differed in their number of movements for each hand. In case 1, the experienced surgeon equally exploited his left and right hands with 811 and 911 (left/right) movements, resulting in a deficit of 100 movements (right/left movements) and a rate of 0.98 and 1.10 movements per second, respectively. The ratio of movements per second was used to eliminate the time duration caused by various reasons during operation. The parameter movement deficit was used to compare the activity of the right hand with that of the left hand. Low ratios are preferred as they indicate greater bimanual dexterity. The inexperienced surgeon performing the surgery in case 6 made many more movements with the right hand that resulted in a deficit of 293 movements and a rate of 1.29 movements per second for the right hand.During dissection of the cystic duct the experienced surgeon made 0.51 movement per second and even fewer during clipping and cutting of the duct (0.39 movements per second). The junior surgeon produced a similar performance of dexterity during cystic duct dissection (0.69 movements per second), though was making more movements during clipping and cutting of the duct (0.84 movements per second).The surgeon in case 9 had to deal with a case of acute cholecystitis, and the entire procedure was completed in 3080.8 seconds. Despite the difficulty, low rates of 0.39 and 0.89 movements per second (left/right) were still achieved.COMMENTTo develop a competency-based system of progression in surgical training, it is necessary to incorporate validated tools of assessment within the program. This pilot study has shown that it is feasible to synchronize dexterity-based and video analysis systems into 1 platform and that this may lead to greater efficiency during the assessment of surgical skill. The ultimate goal is, of course, to completely automate assessment in surgery, though this is a distant notion. Rosen et alhave discussed the introduction of probabilistic methods to surgical skills assessments through the use of mathematical models, with encouraging preliminary results.However, one of the main problems with objectification of any type of surgical skills assessment is the complexity of the procedure and variability in different surgical styles and approaches. That is why it is necessary for expert surgeons to analyze videos of the procedure, ensuring each surgeon is progressing in a safe and purposeful manner. Current distinctions between experienced and novice surgeons involve numbers of procedures performed, though further validation of objective assessment tools can enable the use of dexterity and quality parameters to discriminate between surgeons. It may then be possible to define progression in terms of improvement in dexterity and quality, rather than on the number of procedures performed, or complications experienced.In terms of concurrent hand motion and video analysis as a method of approaching surgical skills assessment, hand movement analysis has been used extensively in other areas of science, including engineering and military applications. In hand sign recognition, hand tracking is used to record path sequences of spatial data and to train decision-making systems. These can be integrated into home appliances and execute commands according to hand signs.A number of studies have been conducted to track parts of the human body and investigate the relationship between kinematics and human anatomy.For example, an electromagnetic device has been used to measure lumbar spinal and pelvic motion of elite level rowers on a rowing ergometer to quantify and discriminate between good and bad rowing styles.Further studies may make it possible to define “ideal” movement sequences to improve performance.Hand motion analysis has also been applied in robotics to understand and mimic natural human hand movements,aeronautics,biomechanics,and handwritten recognition.Movement tracking sensors have been applied to digitally record handwritten scripts and recognize their correspondent alphanumeric characters using probabilistic stochastic approaches such as hidden Markov models and neural networks.Data retrieval by this system is reliable and instantaneous and may be able to reduce the time taken for video-based assessment of technical skills by focusing in on key areas of the operation. Furthermore, parts of the operation that have a more movements-per-unit time may indicate a complication such as bleeding. Our preliminary results have demonstrated the potential use of this system as a promising tool in a real environment. It has been shown that physical and technical constraining factors in an operating theater do not affect data acquisition or data analysis. Therefore, it is our belief that this application should continue to be explored, thus pursuing the maximal potential of this technology.Correspondence:Aristotelis Dosis, MSc, Surgical Computing and Imaging Research Group, Department of Surgical Oncology and Technology, Imperial College London, 10th Floor, Queen Elizabeth, Queen Mother Wing, St Mary’s Hospital, London W2 1NY, United Kingdom (aristotelis.dosis@imperial.ac.uk).Accepted for Publication:September 21, 2004.Funding/Support:The BUPA Foundation, London, England, contributed financially to this research.Acknowledgments:We thank all of the past research fellows of the department for their contribution to the development and validation of ICSAD.REFERENCESRSmithAll changed, changed utterly: British medicine will be transformed by the Bristol case.BMJ1998316191719189641922ADarziSSmithNTaffinderAssessing operative skill: needs to become more objective.BMJ199931888788810102830DMKaufmanMWeinbergerJJStrainJWJacobsDetection of cognitive deficits by a brief mental status examination: the Cognitive Capacity Screening Examination, a reappraisal and a review.Gen Hosp Psychiatry19791247255499797RReznickGRegehrHMacRaeJMartinWMcCullochTesting technical skill via an innovative “bench station” examination.Am J Surg19971732262309124632Southern Surgeons ClubA prospective analysis of 1518 laparoscopic cholecystectomies.N Engl J Med1991324107310781826143JShahADarziSurgical skills assessment: an ongoing debate.BJU Int20018865566011890231VKDattaSMMackayAAChangAWDarziElectromagnetic motion analysis in the assessment of surgical technical skill.ANZ J Surg20027263263412269912GBHannaTDrewACuschieriTechnology for psychomotor skills testing in endoscopic surgery.Semin Laparosc Surg1997412012410401149JAMartinGRegehrRReznickObjective structured assessment of technical skill (OSATS) for surgical residents.Br J Surg1997842732789052454VDattaDFGilliesSMackayADarziMotion analysis in the assessment of surgical skill.Comput Methods Biomech Biomed Eng20014515523SDBannMSKhanAWDarziMeasurement of surgical dexterity using motion analysis of simple bench tasks.World J Surg20032739039412658479VDattaSMackayMMandaliaADarziThe use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model.J Am Coll Surg200119347948511708503SGSmithJTorkingtonTJBrownNJTaffinderADarziMotion analysis: a tool for assessing laparoscopic dexterity in the performance of a laboratory-based laparoscopic cholecystectomy.Surg Endosc20021664064511972205MSKhanSDBannADarziPEButlerUse of suturing as a measure of technical competence.Ann Plast Surg20035030430812800910VDattaAChangSMackayADarziThe relationship between motion analysis and surgical technical assessments.Am J Surg2002184707312135725JDHernandezSDBannYMunzQualitative and quantitative analysis of the learning curve of a simulated surgical task on the da Vinci system.Surg Endosc20041837237814752634YMunzADosisBLoObjective assessment of performance of robotic assisted lima to lad anastomosis on a bench-model.Heart Surg Forum20036(supp 1)S24ADosisFBelloTRockallROVIMAS: a software package for assessing surgical skills using the da Vinci telemanipulator system.Paper presented at: The Fourth International Conference of Information Technology (ITAB 2003); April 24-27, 2003; Birmingham, EnglandADosisFBelloKMoorthyYMunzDGilliesADarziReal-time synchronization of kinematic and video data for the comprehensive assessment of surgical skills.Paper presented at: the 12th Conference on Medicine Meets Virtual Reality; January 15, 2004; Los Angeles, CalifJRosenMMacFarlaneCRichardsBHannafordMSinananSurgeon-tool force/torque signatures evaluation of surgical skills in minimally invasive surgery.Medicine Meets Virtual Reality: The Convergence of Physical & Informational Technologies: Options for a New Era in Healthcare199962290296JRosenMSolazzoBHannafordMSinananObjective laparoscopic skills assessments of surgical residents using hidden Markov models based on haptic information and tool/tissue interactions.Paper presented at: The Ninth Conference on Medicine Meets Virtual Reality; January 24, 2001; Los Angeles, CalifYTCuiJJWengHand sign recognition from intensity image sequences with complex backgrounds.Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.Los Alamitos, Calif: IEEE Computer Society Press; 1996:259-264HWatanabeHHongoMYasumotoYNiwaKYamamotoControl of home appliances using face and hand sign recognition.The Eighth IEEE International Conference on Computer Vision.Vol 2. Los Alamitos, Calif: IEEE Computer Society Press; 2001:746AMulderHuman Movement Tracking Technology.Tech report TR 94-1. 2004. Vancouver, British Columbia: School of Kinesiology, Simon Fraser UniversityAMBullAHMcGregorMeasuring spinal motion in rowers: the use of an electromagnetic device.Clin Biomech (Bristol, Avon)20001577277611050361JSCuiZQSunModel-based visual hand posture tracking for guiding a dexterous robotic hand.Opt Commun2004235311318LMartorellaGDi PietroGDocileMBergamascoRo-Man 2003: 12th IEEE International Workshop on Robot and Human Interactive CommunicationHAnD: a haptic system for analysis and driving of hand movements in augmented reality environment.New York, NY: IEEE Computer Society Press; 2003:171-174YLinWJZhangConference Proceeding: IEEE International Conference on Systems, Man and CyberneticsCombining eye movement and hand movement measures for evaluating human-machine interfaces.Vol 2. New York, NY: IEEE Computer Society Press; 2003:1089-1096KMChungRHSoEffects of hand movement lags on discrete manual control task performance in virtual environments.Paper presented at: 43rd Annual Meeting of the Human-Factors-and-Ergonomics-Society; September 27, 1999; Houston, Tex. Also available at: http://www.hfes.org/Publications/1999proc.htmlRHSoMJGriffinProceedings of Flight Simulation Technologies ConferenceCollection of Technical PapersEffects of time delays on head tracking performance and the benefits of lag compensation by image deflection.New York, NY: ACM; 1991:124-130EVBiryukovaARoby-BramiAAFrolovMMokhtariKinematics of human arm reconstructed from spatial tracking system recordings.J Biomech20003398599510828329NAricaFTYarman-VuralOne-dimensional representation of two-dimensional information for HMM based handwritten recognition.In: Conference Proceedings: IEEE International Conference on Image Processing.Vol 2. New York, NY: IEEE Computer Society Press; 1998:948-952KHanIKSethiHandwritten signature retrieval and identification.Pattern Recognit Lett1996178390
TI - Synchronized Video and Motion Analysis for the Assessment of Procedures in the Operating Theater
JF - JAMA Surgery
DO - 10.1001/archsurg.140.3.293
DA - 2005-03-01
UR - https://www.deepdyve.com/lp/american-medical-association/synchronized-video-and-motion-analysis-for-the-assessment-of-7Sn262Sdfp
SP - 293
EP - 299
VL - 140
IS - 3
DP - DeepDyve
ER -