Measuring surgical safety during minimally invasive surgical procedures: a validation study

Measuring surgical safety during minimally invasive surgical procedures: a validation study Background During the implementation of new interventions (i.e., surgical devices and technologies) in the operating room, surgical safety might be compromised. Current safety measures are insufficient in detecting safety hazards during this process. The aim of the study was to observe whether surgical teams are capable of measuring surgical safety, especially with regard to the introduction of new interventions. Methods A Surgical Safety Questionnaire was developed that had to be filled out directly postoperative by three surgical team members. A potential safety concern was defined as at least one answer between (strongly) disagree and indifferent. The validity of the questionnaire was assessed by comparison with the results from video analysis. Two different observers annotated the presence and effect of surgical flow disturbances during 40 laparoscopic hysterectomies performed between November 2010 and April 2012. Results The surgeon reported a potential safety concern in 16% (85/520 questions). With respect to the scrub nurse and anesthesiologist, this was both 9% (46/520). With respect to the preparation, functioning, and ease of use of the devices in 37.5–47.5% (15–19/40 procedures) a potential safety concern was reported by one or more team members. During procedures after which a potential safety concern was reported, surgical flow disturbances lasted a higher percentage of the procedure duration [9.3 ± 6.2 vs. 2.9 ± 3.7% (mean ± SD), p < .001]. After procedures during which a new instrument or device was used, more potential safety concerns were reported (51.2 vs. 23.1%, p < .001). Conclusions Potential safety concerns were especially reported during procedures in which a relatively high percentage of the duration consisted of surgical flow disturbances and during procedures in which a new instrument or device was used. The Surgical Safety Questionnaire can act as a validated tool to evaluate and maintain surgical safety during minimally invasive procedures, especially during the introduction of a new intervention. Keywords Interobserver reliability · Minimally invasive surgery · Video observation · Surgical safety In the ongoing search for optimal patient outcomes, surgi- cal procedures are continuously evolving [1]. As a result, maintaining the high level of patient safety has become a Electronic supplementary material The online version of this great challenge [2]. Implementing new techniques and/or article (https://doi.org/10.1007/s00464-018-6021-7) contains supplementary material, which is available to authorized users. technologies causes changes in standardized surgical proce- dures to which every surgical team member has to adapt [3, * Mathijs D. Blikkendaal 4]. Monitoring surgical safety in the operating room (OR) is m.d.blikkendaal@lumc.nl one of the most important issues to guarantee optimal surgi- Department of Gynecology, Leiden University Medical cal outcome. However, real-time monitoring of the surgical Center, P.O. Box 9600, 2300 RC Leiden, The Netherlands safety during a procedure is difficult. The question is: what Department of Gynecology, Haaglanden Medical Center, and how should we monitor and who should do it? P.O. Box 96900, 2509 JH The Hague, The Netherlands Previous studies describing patient safety during mini- Department of BioMechanical Engineering, Delft mally invasive surgery (MIS) have defined certain domains University of Technology, Mekelweg 2, 2628 CD Delft, that are ‘at risk’ [5–8]. In daily practice the identification The Netherlands Vol.:(0123456789) 1 3 3088 Surgical Endoscopy (2018) 32:3087–3095 of these safety issues is often limited to observers that were observation was based on video registration of the proce- physically present in the OR and retrospective interpretation dures. Outcome measures were the number, types, effect, of the obtained data [6, 9, 10]. Adequate interpretation is dif- and duration of surgical flow disturbances per procedure. ficult and requires correct differentiation of errors (undesired The LH was chosen as procedure of interest, because actions) from events (consequence of undesired actions) [5]. it is an advanced laparoscopic procedure performed by a Currently, patient safety indicators are frequently based on dedicated operating team and requiring a wide array of the occurrence of adverse events [11]. However, in general, endoscopic instruments and equipment. The study started intraoperative adverse events rarely occur. In theory, for an in November 2010 and all consecutive LHs that were per- adverse event to occur several errors have to line up and slip formed in a conventional (cart-based) OR were registered through the holes of existing safety barriers [12]. Usually until the start of the construction of the new integrated OR most errors that precede a potential adverse event are timely (Karl Storz OR1™ integrated OR system, September 2011). recognized and dealt with. However, these near-misses dis- After construction of the integrated OR (October 2011), the turb the surgical flow to a greater or lesser extent and there- same amount of eligible procedures was registered in this fore interfere with surgical safety [3–5, 10, 13–16]. setting. Similarly, the occasional introduction of new devices In daily practice, there is no external observer present in both the conventional and integrated OR was registered. during a procedure. The only ‘real-time monitoring’ of In this manner, not only the transition to the integrated OR, patient safety is done by the surgeon and/or the entire sur- but also the introduction of new devices was analyzed. All gical team itself. However, from a psychological perspec- procedures were performed by either of the two gynecolo- tive it is known that an individuals’ situational awareness gists with more than 10 years of experience in advanced is impaired when occupied with a (difficult) task [17]. gynecologic laparoscopy and were assisted by one gynecolo- Regarding this phenomenon, implementing new surgical gist who conducted a fellowship in MIS; a group of five devices and technologies in the OR puts more pressure on alternated in the position of either circulating or scrub nurse. the responsibility of the surgeon to maintain surgical safety The study was approved by the Executive Board of the during the whole procedure [1, 15]. The only measures to Haaglanden Medical Center. Prior to the start of the study, enhance safety throughout a procedure that currently are—or all OR personnel were collectively informed about the study. at least should be—used, are the preoperative team briefings, From each patient, informed consent was obtained. This the postoperative debriefings and, to a lesser extent, some design was adapted from another study [3]. preoperative checklists. In general, these safety instruments have proven to diminish preventable errors during the pro- Development of Surgical Safety Questionnaire cedure and to safeguard open communication [18–21]. How- ever, since these tools do not incorporate items to evaluate Patient safety risk factors that have been described by Rod- new surgical techniques or technologies, they are insufficient rigues et al., were summarized in a questionnaire consist- in detecting safety hazards during their introduction. ing of 13 questions (i.e., time-out/sign-out, preparation and Therefore, the aim of this study was to observe whether functioning of devices and instruments, functioning of the surgical teams are capable of measuring surgical safety, surgical team, distracting stimuli, and interference of the especially with regard to the introduction of new techniques study on the procedure) [6]. Directly after each procedure and technologies during a series of MIS procedures. A the (assisting-)surgeon, scrub nurse, and anesthetist(-assis- questionnaire that had to be filled out directly postoperative tant) filled out this short questionnaire. Answers were given was developed to measure surgical safety. Next, the validity on a 5-point Likert scale, ranging from (strongly) disagree of the questionnaire was assessed by comparison with the to (strongly) agree. A potential safety concern was defined results from independent video analysis of these procedures. as an answer between (strongly) disagree and indifferent by at least one member of the surgical team. Additionally, several questions regarding experience (with the procedure, Materials and methods laparoscopy in general, and the used instruments/devices) and the procedure (adhesions, adverse events) were stated In a university-affiliated teaching hospital (Haaglanden Med- (see Online Appendix). ical Center, The Hague), a prospective registration study was set-up to record and analyze surgical flow disturbances. Video analysis During a consecutive series of laparoscopic hysterectomies (LH), a questionnaire was filled out in the OR by the surgical The input from three video signals (endoscopic image and team members. The surgical flow disturbances were scored two dome cameras) and four audio signals (MPEG Recorder by an independent observer. To minimize the interference 2.1) was synchronously recorded during all procedures. The of the study on its own results (the ‘Hawthorne effect’), this recordings were started just before the time-out procedure 1 3 Surgical Endoscopy (2018) 32:3087–3095 3089 and stopped after suturing all port-sites. The procedure was Statistics excluded from analysis in case of technical problems related to the recording equipment. Two residents in Obstetrics and To facilitate statistical analysis, the recordings were Gynecology (M.D.B. and S.R.C.D.) analyzed the presence annotated with The Observer® XT 11.5 software (Nol- and effect of predefined surgical flow disturbances. These dus Information Technologies, Wageningen, The Nether- surgical flow disturbances were defined as stimuli distract- lands). To assess the interobserver variability, a random ing one or more members of the surgical team (Table 1). To sample of six recordings was scored by both observers. assess the severity, the effect of the surgical flow disturbance The findings of the two observers for these six procedures on the surgical team members was graded according to a were compared and the interobserver agreement was cal- seven-point scale. This scale ranges from 1 as a potentially culated (compares events between two observations and distracting stimulus to 7 when the sterile team’s work is takes the frequency and sequence into account; function completely interrupted (modified by Persoon et al. originally incorporated in The Observer® XT 11.5 software). After described by Healey et al.) (Table 2) [9, 22]. satisfactory interobserver agreement was achieved, the remaining procedures were annotated by either one of the two observers (randomly allocated and analyzed in a non-chronological random order) [23, 24]. For statisti- Table 1 Observed types of surgical flow disturbances cal analysis, SPSS 23 statistical software was used. Intra- class correlation coefficient (ICC) was used to assess the Equipment-/instrument-related inter-rater agreement. A two-way random effects model  Set-up device/connection was used since both the procedures as well as the raters  Intraoperative repositioning are a random sample from a larger pool of procedures  Malfunctioning and raters. We checked for consistency (i.e., raters have  Not present a similar pattern of scores). Outcomes are both average  Sterility measures and single measures. Average measures provide  Other/unclear the reliability of the score being able to separate different Environmental levels of safety, despite differences in individual scoring.  Pager/telephone Single Measures represent the reliability you would get if  Door washing room one rater was used. Values between 0.4 and 0.75 were con-  Radio use sidered to represent “fair to good reliability” and > 0.75 Personnel-related “excellent reliability” [25]. In case the kappa becomes  Communication failure negative (due to low variability and high agreement), the  Irrelevant conversation absolute agreement was described as a percentage [26]. Procedure-related A Pearson Chi-square test was used to compare propor-  Extra coagulation bleeding-site tions and a Mann–Whitney U test was used for continuous  Unexpected adhesions variables (non-normally distributed data). A p < .05 was  Limited vision (condensation/smoke) considered statistically significant.  Adverse event  Conversion to laparotomy Table 2 Effect of observed surgical flow disturbances (according to Persoon et al. [9]) 1 Events with the potential to distract the sterile team 2 Sterile team member momentarily distracted: possible involvement of a single sterile member in an event not related to the primary task, e.g., a short head turn in response to a visual or auditory stimulus 3 Sterile team member engages in distraction: similar distraction in 2, but the sterile member engages with the source of distraction by verbally responding while maintaining primary task activity (multitasking) 4 Sterile team member’s primary task interrupted: a single team member ceases his/her current tasks to engage entirely in the distracting stimu- lus 5 Sterile team momentarily distracted: two or more sterile team members respond to a stimulus with a short head turn, no verbal response 6 Sterile team engage in secondary tasks: two or more team members engage with the source of distraction by verbally responding while main- taining primary task activity 7 Sterile team’s work interrupted—operation flow disrupted: interruption of the current primary task of the sterile team, the operation flow is disrupted 1 3 3090 Surgical Endoscopy (2018) 32:3087–3095 Table 3 Patient and procedure characteristics of analyzed LHs per- Results formed in the Haaglanden Medical Center, The Hague, between Janu- ary 2011 and April 2012 During the study period, 84 LHs were performed of which Overall (N = 40) 40 were eligible for inclusion in two studies [3]. For detailed information on the excluded procedures, see Fig. 1. All pro- Median IQR Min–max cedures were successfully completed and three minor post- Age (years) 48.2 43.9–55.2 operative complications were noted (Tables 3, 4). BMI (kg/m ) 24.9 22.7–27.3 The (assisting-)surgeon answered 95% of all questions Uterine weight (g) 165 97–256 [494 out of total 520 questions (40 procedures, 13 ques- Operating time (min) 121 ± 29 66–176 tions per procedure)], the scrub nurse answered 89% (461 Procedure time (min) 156 ± 31 98–215 out of 520), and the anesthetist(-assistant) answered 86% Estimated blood loss (mL) 100 50–175 of the questions (445 out of 520). Based on the question- Hospital stay (days) 2.0 1.1–2.1 naire, all surgical team members were of the opinion that Benign indication (%) 70.0% the study did not interfere with the procedure in 33 out of the 40 procedures (83%). In all cases, one of the two expe- IQR inter quartile range (25th and 75th percentile), BMI Body Mass Index rienced gynecologists (> 100 LHs) attended the procedure. Time between first incision and last suture (skin-to-skin) (based on Nevertheless, the questionnaire was filled out in 58% of video observation) the cases by the assisting surgeon. As a result, reported Time between patient entering OR and leaving OR (based on video experience of the surgeon with LH varied between ≤ 25 observation) prior procedures (14%), 26–40 (30%), 41–100 (32%), and > 100 prior LHs in 24% of the procedures. The surgeons MIS was in 37% of the cases between 41 and 100 and reported in 41% of the cases to have used the same instru- in 53% > 100 prior procedures. Despite this, experience ments and devices > 100 times before in prior procedures. with LH specifically was moderate; in 71% of the cases In 50% they reported to have experience with the equip- the scrub nurse had performed ≤ 25 prior LH procedures. ment between 25 and 100 prior procedures and in 8% this Similarly, their experience with the equipment was moder- was ≤ 25 procedures. Experience of the scrub nurse with ate (in 43–47% of the cases ≤ 25 procedures). Total performed procedures during study period N=84 Procedures in Procedures in other integrated Procedures in conventional OR OR (during construction) integrated OR N=11 N=27 N=46 Excluded: 4 no informed consent Excluded: 5 problem video recording Excluded: 3 no informed consent 6 problem audio recording 11 not registered 2 technical failure 3 other reasons 2 other reasons First 8 procedures due to maximum of 20 reached Eligible recorded procedures Eligible recorded procedures conventional OR integrated OR N=20 N=20 Total recordings used for analysis N=40 Fig. 1 Inclusion of eligible procedures 1 3 Surgical Endoscopy (2018) 32:3087–3095 3091 Table 4 Adverse events all analyzed LHs In general, scores given by the surgeon were in 16% (85/520) regarded as a potential safety concern. With respect Overall (N = 40) to the scrub nurse and anesthesiologist this was both 9% Infection 1 (2.5%) (46/520). Overall, ‘strongly disagree’ was reported in 2% Blood loss > 1L 1 (2.5%) (9/520), of which 8 were reported on questions 3, 4, or 5 Others 1 (2.5%) (i.e., equipment related, see Online Appendix). Total 3 (7.5%) In 87% (452 of 520 questions), all members of the surgi- cal team agreed in their answers (i.e., the maximum differ - All adverse events did not require re-operation and occurred postop- ence between the lowest and the highest was ≤ one point on eratively the Likert scale). In 4% (22 of 520), the absolute difference LH Laparoscopic hysterectomy between the members of the surgical team was high (≥ 3; for Urinary tract infection example, to the same question the surgeon reports ‘disagree’ Postoperative drop in hemoglobin. CT-scan showed free fluid intra- abdominally. Vital signs were stable and after a blood transfusion and the scrub nurse reports ‘strongly agree’). The ICC was with two packed cells hemoglobin levels remained stable 0.44 (average measures). Patient suffered from sensibility loss in her right hand. The neurolo- gist diagnosed a neurapraxia of the median nerve. Conservative man- Validation of Surgical Safety Questionnaire by video agement resulted in almost complete recovery analysis The overall observation duration of these procedures was Surgical Safety Questionnaire 103 h and 45 min. Six randomly chosen observations were annotated by both observers and showed excellent agree- The scores per question of the individual team members are summarized in Table  5. In 15% (6 out of 40) of the ment (Cohen’s Kappa of 0.79–0.98, all observations com- bined 0.85, p < .001). Therefore, the remaining procedures procedures, potential safety concerns [i.e., answer ‘indif- ferent’ or ‘(strongly) disagree’] were reported regarding were annotated by the two observers separately (in total 36 observations by M.D.B. and 10 by S.R.C.D., respectively). the time-out and sign-out procedure. With respect to the preparation, functioning, and ease of use of the devices The duration and effect of disturbances during procedures in which a potential safety concern was reported with regard in 37.5–47.5% (15–19 out of 40 procedures) a potential safety concern was reported by one or more team mem- to the functioning of devices and instruments (question 4, see Online Appendix) were compared to the procedures in bers. A strong disagreement to a flawless use of the devices was reported in seven procedures (17.5%). With respect to which no safety concern was reported (Table 6). In the pro- cedures after which a potential safety concern was reported, communication and collaboration in 30–35% (12–14 out of 40 procedures) concerns were reported, mostly by the a significantly higher percentage of the duration of the pro- cedure consisted of surgical flow disturbances [9.3 ± 6.2 vs. surgeon. Table 5 scores per question of the team members individually Question Surgeon Scrub nurse Anesthetist N Mean ± SD Range N ≤ 3 N Mean ± SD Range N ≤ 3 N Mean ± SD Range N ≤ 3 Time-out 39 4.54 ± 0.55 3–5 1 36 4.19 ± 0.67 2–5 3 37 4.08 ± 0.68 2–5 3 Sign-out 37 4.49 ± 0.51 4–5 0 31 4.16 ± 0.86 2–5 5 28 3.96 ± 0.51 2–5 2 Preparation 39 3.97 ± 1.06 1–5 11 36 4.14 ± 0.72 2–5 5 34 3.88 ± 0.81 2–5 7 Functioning 39 3.51 ± 1.21 1–5 16 36 3.83 ± 1.11 1–5 6 33 3.85 ± 0.67 2–5 6 Ease of use 39 3.82 ± 1.07 1–5 11 36 3.94 ± 0.83 2–5 5 32 3.81 ± 0.74 1–5 7 Communication 39 3.9 ± 0.75 2–5 11 35 3.86 ± 0.77 2–5 5 36 4.11 ± 0.52 3–5 3 Collaboration 39 3.92 ± 0.74 2–5 10 36 3.89 ± 0.62 2–5 5 36 4.14 ± 0.42 3–5 1 Disturbances 39 3.95 ± 0.92 2–5 7 36 3.89 ± 0.85 1–5 4 35 3.77 ± 0.81 2–5 8 Surgeon 28 3.96 ± 0.43 3–5 3 36 4.25 ± 0.55 3–5 2 35 4.14 ± 0.49 3–5 2 Scrub nurse 39 3.92 ± 0.62 2–5 7 35 4 ± 0.48 3–5 4 35 4.14 ± 0.43 3–5 1 Anesthetist 39 4.18 ± 0.51 3–5 2 36 4.19 ± 0.47 3–5 1 32 4.41 ± 0.5 4–5 0 Patient safety 39 4.21 ± 0.7 3–5 4 36 4.08 ± 0.5 2–5 1 36 4.42 ± 0.5 4–5 0 Study influence 39 4.56 ± 0.6 3–5 2 36 4.31 ± 0.47 4–5 0 36 3.97 ± 0.81 2–5 6 N ≤ 3: The number of questions to which a score ≤ 3 was given, which is defined as a safety concern 1 3 3092 Surgical Endoscopy (2018) 32:3087–3095 Table 6 Duration and effect of surgical flow disturbances with regard to functioning of devices and instruments (question 4 of questionnaire) separated between procedures with or without a safety concern reported by at least one member of the surgical team (N = 40 procedures) d d No safety concern reported Safety concern reported N Mean ± SD Min–max N Mean ± SD Min–max p Percentage of procedure 21 2.9 ± 3.7 0.0–15.4 19 9.3 ± 6.2 1.6–21.7 < .001 Effect (weighted) 21 4.4 ± 2.4 0.0–7.0 19 6.1 ± 1.9 3.0–7.0 .020 Impact 21 13.2 ± 12.0 0.0–47.1 19 56.2 ± 38.7 11.5–145.7 < .001 SD standard deviation Total duration of the disturbance (s) defined as percentage of the total procedure time Effect of the disturbance (based on Persoon et al. [9]) corrected by the duration of the disturbance(s) Percentage of procedure multiplied by weighted effect Reported answer by at least one surgical team member was (strongly) disagree or indifferent Mann–Whitney U test for independent samples 2.9 ± 3.7% (mean ± SD), p < .001]. Similarly, in these proce- suture for the vaginal cu ff (one procedure), and multiple new dures, a significantly higher mean weighted effect (i.e., the devices/instruments (three procedures). mean effect of the disturbances corrected for the duration of the disturbances) was found (score 6.1 ± 1.9 vs. 4.4 ± 2.4, Experience p = .020; see Table 2 for the meaning of the scores). In the group without any reported safety concerns, there Limited experience of the scrub nurse with the equipment were only two procedures during which a relatively high (≤ 25 procedures) resulted in significantly more potential percentage of the procedure consisted of disturbances (10.0 safety concerns reported by at least one member of the sur- and 15.4%, respectively). However, the mean weighted effect gical team (30.7 vs. 15.6%, p = .002). However, this did not of these disturbances was low (1.9 and 3.0, respectively) and result in a higher percentage of procedure time expended to therefore can be regarded as adequately managed. All tests to surgical flow disturbances (7.3 ± 7.6 vs. 5.0 ± 5.2%, p = .423) assess whether using the questionnaire of one or two of the and/or a higher effect of these disturbances (5.7 ± 1.4 vs. team members might be applicable as well resulted in lower 4.8 ± 2.3, p = .275) (N = 30 procedures). Experience of the agreement with the video analysis (not shown). surgeon with the used instruments did not have a significant influence on the potential safety concerns either (25.6 vs. Newly introduced devices and/or technology 23.8%, p = .791). During eight procedures (20%, four procedures in the con- ventional OR and four in the integrated OR), a new instru- Discussion ment and/or device was used. During these procedures, the surgical team members reported a potential safety concern in The Surgical Safety Questionnaire filled out directly post - 51% (41 out of 80 questions regarding intraoperative aspects operative by all members of the surgical team proved to be (question 3 till 12), see Online Appendix). In contrast, the a valid tool to adequately estimate surgical safety in MIS. prevalence of a potential safety concern during the other Procedures during which a relatively high percentage of the procedures was 23.1% (74 out of 320, p < .001). duration consisted of surgical flow disturbances and/or with The first 20 procedures were performed in a conventional a relatively high mean weighted effect of these disturbances cart-based OR. The last 20 procedures were performed in a matched with the reported potential safety concerns. Fur- new integrated OR. No difference in potential safety con- thermore, during procedures in which a new instrument or cerns was reported between the two OR set-ups (28 vs. 29%, device was used, significantly more potential safety concerns p = .740). Furthermore, an employee of the medical industry were reported by the surgical team. Therefore this could be was present during seven procedures (four in conventional a useful tool in the evaluation and maintenance of surgical OR, three in integrated OR), during which a newly intro- safety during the introduction of new surgical equipment or duced device was used. Additionally, in one procedure a technology. new device was used without an employee of the industry The term patient safety is at risk to become an empty being present (fourth consecutive procedure in which this phrase by its broad interpretation. To define nuances in instrument was used). The new equipment concerned a new patient safety, the ‘systems approach’ is most commonly bipolar sealing instrument (five procedures), a new type of used [27, 28]. Based on this approach, several studies 1 3 Surgical Endoscopy (2018) 32:3087–3095 3093 introduced frameworks covering the risk domains relevant Furthermore, in contrast to the high agreement (87%), the to surgical safety and patient outcomes [6, 7, 29]. The reported ICC (0.44) seems low. However, this discrepancy questionnaire validated in present study covers these risk is explained by the low variability and high agreement in domains and thereby provides a composite outcome for sur- the reported answers. In those cases, kappa is not a reliable gical safety. estimate for correlation [26]. Thirdly, the reported experi- A study conducted by Russ et al. had similar objectives ence with the LH seems low. This is due to the system in and described the Metric for Evaluating Task Execution in The Netherlands, in which residents specializing in MIS the Operating Room (METEOR) as an easy to use tool to are usually allowed to perform LH as ‘primary’ surgeon allow surgical teams to self-assess their performance, in during the last year of their residency and therefore also order to track surgical hazards, and to be able to evaluate filled out our scoring sheets. However, without exception, safety [30]. However, their checklist is quite extensive (up in these cases, the senior consultant with extensive expe- to 80 items) and does not cover concerns regarding instru- rience in advanced gynecologic endoscopy was always ments and devices. Since the high dependency on technol- member of the sterile team as well. ogy in MIS, equipment-related disturbances are one of the Over the past decades patient outcomes regarding MIS well-known primary sources of disruption [3, 8, 31]. Addi- have rapidly improved. Large leaps could be made in the tionally, during the introduction of a new technique and/or early days of MIS, where measures taken to improve safety technology in the OR, disruptions are even more likely to were highly effective. Currently, only smaller steps can be occur [4, 7]. This hazard is also one of the main results in made with a higher risk of doing harm instead of good [1, our study. Therefore, prior to the introduction of a new inter- 35]. Furthermore, the OR has become increasingly com- vention in the OR, a prospective risk analysis should be per- plex. As Sir Cyril Chantler said: “Medicine used to be formed to guarantee safe implementation (e.g., Healthcare simple, ineffective and relatively safe. Now it is complex, Failure Mode and Effect Analysis) [32]. Nevertheless, in effective and potentially dangerous” [36]. The common our opinion, methods currently used to monitor this imple- objective we are pursuing is to enable technology to assist mentation (i.e., evaluation after 6 and 12 months, adverse the surgeon and its team in maintaining surgical safety. events registration, incident reporting system) fail to detect Similar to recent developments in the automotive industry safety concerns in a timely manner. Similarly, our results to assist the driver on traffic safety (e.g., collision avoid- rule out the widespread assumption that an employee of the ance, blind spot detection, and lane departure warning medical industry being present can prevent safety hazards. systems), some promising systems are currently tested in Instead, the Surgical Safety Questionnaire presented in this a few hospitals in The Netherlands. For example, the Digi- study could be a useful tool to systematically evaluate the tal Operating Room Assistant continuously monitors the surgical safety after each procedure, especially in case of the location, status, and (mal)functioning of devices [37, 38]. introduction of a new instrument or technology. In conclusion, the results of our study demonstrate that The main strength of our study is that by using video the presented Surgical Safety Questionnaire can act as a observation we were able to assess surgical flow distur - validated tool to evaluate and maintain surgical safety dur- bances without influencing the course of the procedure. ing minimally invasive procedures. In daily practice, we In that way, we obtained very reliable quantitative results recommend to fill out this questionnaire in case a new to serve as gold standard and thereby allowing validation technique or technology is used during a procedure. By of our Surgical Safety Questionnaire. This is in line with involving the complete surgical team with their individual other studies recognizing the additional value of detailed knowledge, experience, and opinions, this will provide analysis of video registration [33, 34]. A weakness could the opportunity to constantly evaluate new equipment and be that scoring on a 5-point Likert scale remains prone techniques. As a consequence, in an early stage, potential to subjectivity. What determines the difference between safety hazards will be prevented in future patients. agree, neither agree nor disagree, and disagreement? It Acknowledgements The authors want to acknowledge Arjan van Dijke was decided to place the cut-off for a potential safety con - for his extensive help with the video observation system. cern at ‘neither agree nor disagree.’ By doing so, every time at least one of the team members for any reason had Compliance with ethical standards a motive to not (fully) agree on a certain question in the questionnaire, the item was marked as potential safety con- Disclosures Mathijs Blikkendaal, Sara Driessen, Sharon Rodrigues, cern. Nevertheless, the results of our study indicate that by Johann Rhemrev, Maddy Smeets, Jenny Dankelman, John Dobbelsteen, and Frank Willem Jansen have no conflicts of interest or financial ties using this definition the potential safety concerns correlate to disclose. very well with the observed surgical flow disturbances. 1 3 3094 Surgical Endoscopy (2018) 32:3087–3095 Open Access This article is distributed under the terms of the Creative 17. Johnson HL, Kimsey D (2012) Patient safety: break the silence. Commons Attribution 4.0 International License (http://creativecom- AORN J 95:591–601 mons.org/licenses/by/4.0/), which permits unrestricted use, distribu- 18. Treadwell JR, Lucas S, Tsou AY (2014) Surgical checklists: a tion, and reproduction in any medium, provided you give appropriate systematic review of impacts and implementation. BMJ Qual Saf credit to the original author(s) and the source, provide a link to the 23:299–318 Creative Commons license, and indicate if changes were made. 19. Gluck PA (2012) Patient safety: some progress and many chal- lenges. Obstet Gynecol 120:1149–1159 20. de Vries EN, Prins HA, Crolla RM, den Outer AJ, van Andel G, van Helden SH, Schlack WS, van Putten MA, Gouma DJ, Dijk- References graaf MG, Smorenburg SM, Boermeester MA (2010) Effect of a comprehensive surgical safety system on patient outcomes. New 1. Eisenberg D, Wren SM (2014) Innovation in safety, and safety in Engl J Med 363:1928–1937 innovation. JAMA Surg 149:7–9 21. Birkmeyer JD (2012) Progress and challenges in improving surgi- 2. Sevdalis N, Hull L, Birnbach DJ (2012) Improving patient safety cal outcomes. Br J Surg 99:1467–1469 in the operating theatre and perioperative care: obstacles, inter- 22. Healey AN, Primus CP, Koutantji M (2007) Quantifying distrac- ventions, and priorities for accelerating progress. Br J Anaesth tion and interruption in urological surgery. Qual Saf Health Care 109(Suppl 1):i3–i16 16:135–139 3. Blikkendaal MD, Driessen SR, Rodrigues SP, Rhemrev JP, Smeets 23. Buzink SN, van LL, de Hingh, Jakimowicz IH JJ (2010) Risk- MJ, Dankelman J, van den Dobbelsteen JJ, Jansen FW (2017) Sur- sensitive events during laparoscopic cholecystectomy: the influ- gical flow disturbances in dedicated minimally invasive surgery ence of the integrated operating room and a preoperative checklist suites: an observational study to assess its supposed superiority tool. Surg Endosc 24:1990–1995 over conventional suites. Surg Endosc 31:288–298 24. Zheng B, Martinec DV, Cassera MA, Swanstrom LL (2008) A 4. Antoniadis S, Passauer-Baierl S, Baschnegger H, Weigl M (2014) quantitative study of disruption in the operating room during lapa- Identification and interference of intraoperative distractions and roscopic antireflux surgery. Surg Endosc 22:2171–2177 interruptions in operating rooms. J Surg Res 188:21–29 25. Fleiss JL (1986) The design and analysis of clinical experiments. 5. Bonrath EM, Dedy NJ, Zevin B, Grantcharov TP (2013) Defining Wiley, New York technical errors in laparoscopic surgery: a systematic review. Surg 26. Feinstein AR, Cicchetti DV (1990) High agreement but low kappa: Endosc 27:2678–2691 I. The problems of two paradoxes. J Clin Epidemiol 43:543–549 6. Rodrigues SP, Wever AM, Dankelman J, Jansen FW (2012) Risk 27. Vincent C, Moorthy K, Sarker SK, Chang A, Darzi AW (2004) factors in patient safety: minimally invasive surgery versus con- Systems approaches to surgical quality and safety: from concept ventional surgery. Surg Endosc 26:350–356 to measurement. Ann Surg 239:475–482 7. Driessen SRC, Sandberg EM, Rodrigues SP, van Zwet EW, Jansen 28. Dankelman J, Grimbergen CA (2005) Systems approach to reduce FW (2017) Identification of risk factors in minimally invasive sur - errors in surgery. Surg Endosc 19:1017–1021 gery: a prospective multicenter study. Surg Endosc 31:2467–2473 29. Calland JF, Guerlain S, Adams RB, Tribble CG, Foley E, Chekan 8. Weerakkody RA, Cheshire NJ, Riga C, Lear R, Hamady MS, EG (2002) A systems approach to surgical safety. Surg Endosc Moorthy K, Darzi AW, Vincent C, Bicknell CD (2013) Surgi- 16:1005–1014 cal technology and operating-room safety failures: a systematic 30. Russ S, Arora S, Wharton R, Wheelock A, Hull L, Sharma E, review of quantitative studies. BMJ Qual Saf 22:710–718 Darzi A, Vincent C, Sevdalis N (2013) Measuring safety and effi- 9. Persoon MC, Broos HJ, Witjes JA, Hendrikx AJ, Scherpbier AJ ciency in the operating room: development and validation of a (2011) The effect of distractions in the operating room during metric for evaluating task execution in the operating room. J Am endourological procedures. Surg Endosc 25:437–443 Coll Surg 216:472–481 10. Parker SE, Laviana AA, Wadhera RK, Wiegmann DA, Sundt TM 31. Verdaasdonk EG, Stassen LP, van der Elst M, Karsten TM, III (2010) Development and evaluation of an observational tool Dankelman J (2007) Problems with technical equipment dur- for assessing surgical flow disruptions and their impact on surgical ing laparoscopic surgery. An observational study. Surg Endosc performance. World J Surg 34:353–361 21:275–279 11. Pronovost PJ, Thompson DA, Holzmueller CG, Lubomski LH, 32. Linkin DR, Sausman C, Santos L, Lyons C, Fox C, Aumiller Morlock LL (2005) Defining and measuring patient safety. Crit L, Esterhai J, Pittman B, Lautenbach E (2005) Applicability of Care Clin 21:1–19 healthcare failure mode and effects analysis to healthcare epide- 12. Reason J (2004) Beyond the organisational accident: the need for miology: evaluation of the sterilization and use of surgical instru- “error wisdom” on the frontline. Qual Saf Health Care 13(Suppl ments. Clin Infect Dis 41:1014–1019 2):ii28–ii33 33. Bezemer J, Cope A, Korkiakangas T, Kress G, Murtagh G, Wel- 13. Wiegmann DA, ElBardissi AW, Dearani JA, Daly RC, Sundt TM don SM, Kneebone R (2017) Microanalysis of video from the III (2007) Disruptions in surgical flow and their relationship to operating room: an underused approach to patient safety research. surgical errors: an exploratory investigation. Surgery 142:658–665 BMJ Qual Saf 26:583–587 14. Weigl M, Antoniadis S, Chiapponi C, Bruns C, Sevdalis N (2015) 34. Makary MA (2013) The power of video recording: taking quality The impact of intra-operative interruptions on surgeons’ perceived to the next level. JAMA 309:1591–1592 workload: an observational study in elective general and orthope- 35. Parsons JK, Messer K, Palazzi K, Stroup SP, Chang D (2014) dic surgery. Surg Endosc 29:145–153 Diffusion of surgical innovations, patient safety, and minimally 15. Pluyter JR, Buzink SN, Rutkowski AF, Jakimowicz JJ (2010) Do invasive radical prostatectomy. JAMA Surg 149:845–851 absorption and realistic distraction influence performance of com- 36. Chantler C (1999) The role and education of doctors in the deliv- ponent task surgical procedure? Surg Endosc 24:902–907 ery of health care. Lancet 353:1178–1181 16. Hsu KE, Man FY, Gizicki RA, Feldman LS, Fried GM (2008) 37. Guedon AC, Wauben LS, Overvelde M, Blok JH, van der Elst M, Experienced surgeons can do more than one thing at a time: Dankelman J, van den Dobbelsteen JJ (2014) Safety status system effect of distraction on performance of a simple laparoscopic and for operating room devices. Technol Health Care 22:795–803 cognitive task by experienced and novice surgeons. Surg Endosc 22:196–201 1 3 Surgical Endoscopy (2018) 32:3087–3095 3095 38. Henken KR, Jansen FW, Klein J, Stassen LP, Dankelman J, van recording in clinical practice. Surg Endosc 26:2909–2916 den Dobbelsteen JJ (2012) Implications of the law on video 1 3 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Surgical Endoscopy Springer Journals

Measuring surgical safety during minimally invasive surgical procedures: a validation study

Free
9 pages
Loading next page...
 
/lp/springer_journal/measuring-surgical-safety-during-minimally-invasive-surgical-60n8abis0Z
Publisher
Springer US
Copyright
Copyright © 2018 by The Author(s)
Subject
Medicine & Public Health; Surgery; Gynecology; Gastroenterology; Hepatology; Proctology; Abdominal Surgery
ISSN
0930-2794
eISSN
1432-2218
D.O.I.
10.1007/s00464-018-6021-7
Publisher site
See Article on Publisher Site

Abstract

Background During the implementation of new interventions (i.e., surgical devices and technologies) in the operating room, surgical safety might be compromised. Current safety measures are insufficient in detecting safety hazards during this process. The aim of the study was to observe whether surgical teams are capable of measuring surgical safety, especially with regard to the introduction of new interventions. Methods A Surgical Safety Questionnaire was developed that had to be filled out directly postoperative by three surgical team members. A potential safety concern was defined as at least one answer between (strongly) disagree and indifferent. The validity of the questionnaire was assessed by comparison with the results from video analysis. Two different observers annotated the presence and effect of surgical flow disturbances during 40 laparoscopic hysterectomies performed between November 2010 and April 2012. Results The surgeon reported a potential safety concern in 16% (85/520 questions). With respect to the scrub nurse and anesthesiologist, this was both 9% (46/520). With respect to the preparation, functioning, and ease of use of the devices in 37.5–47.5% (15–19/40 procedures) a potential safety concern was reported by one or more team members. During procedures after which a potential safety concern was reported, surgical flow disturbances lasted a higher percentage of the procedure duration [9.3 ± 6.2 vs. 2.9 ± 3.7% (mean ± SD), p < .001]. After procedures during which a new instrument or device was used, more potential safety concerns were reported (51.2 vs. 23.1%, p < .001). Conclusions Potential safety concerns were especially reported during procedures in which a relatively high percentage of the duration consisted of surgical flow disturbances and during procedures in which a new instrument or device was used. The Surgical Safety Questionnaire can act as a validated tool to evaluate and maintain surgical safety during minimally invasive procedures, especially during the introduction of a new intervention. Keywords Interobserver reliability · Minimally invasive surgery · Video observation · Surgical safety In the ongoing search for optimal patient outcomes, surgi- cal procedures are continuously evolving [1]. As a result, maintaining the high level of patient safety has become a Electronic supplementary material The online version of this great challenge [2]. Implementing new techniques and/or article (https://doi.org/10.1007/s00464-018-6021-7) contains supplementary material, which is available to authorized users. technologies causes changes in standardized surgical proce- dures to which every surgical team member has to adapt [3, * Mathijs D. Blikkendaal 4]. Monitoring surgical safety in the operating room (OR) is m.d.blikkendaal@lumc.nl one of the most important issues to guarantee optimal surgi- Department of Gynecology, Leiden University Medical cal outcome. However, real-time monitoring of the surgical Center, P.O. Box 9600, 2300 RC Leiden, The Netherlands safety during a procedure is difficult. The question is: what Department of Gynecology, Haaglanden Medical Center, and how should we monitor and who should do it? P.O. Box 96900, 2509 JH The Hague, The Netherlands Previous studies describing patient safety during mini- Department of BioMechanical Engineering, Delft mally invasive surgery (MIS) have defined certain domains University of Technology, Mekelweg 2, 2628 CD Delft, that are ‘at risk’ [5–8]. In daily practice the identification The Netherlands Vol.:(0123456789) 1 3 3088 Surgical Endoscopy (2018) 32:3087–3095 of these safety issues is often limited to observers that were observation was based on video registration of the proce- physically present in the OR and retrospective interpretation dures. Outcome measures were the number, types, effect, of the obtained data [6, 9, 10]. Adequate interpretation is dif- and duration of surgical flow disturbances per procedure. ficult and requires correct differentiation of errors (undesired The LH was chosen as procedure of interest, because actions) from events (consequence of undesired actions) [5]. it is an advanced laparoscopic procedure performed by a Currently, patient safety indicators are frequently based on dedicated operating team and requiring a wide array of the occurrence of adverse events [11]. However, in general, endoscopic instruments and equipment. The study started intraoperative adverse events rarely occur. In theory, for an in November 2010 and all consecutive LHs that were per- adverse event to occur several errors have to line up and slip formed in a conventional (cart-based) OR were registered through the holes of existing safety barriers [12]. Usually until the start of the construction of the new integrated OR most errors that precede a potential adverse event are timely (Karl Storz OR1™ integrated OR system, September 2011). recognized and dealt with. However, these near-misses dis- After construction of the integrated OR (October 2011), the turb the surgical flow to a greater or lesser extent and there- same amount of eligible procedures was registered in this fore interfere with surgical safety [3–5, 10, 13–16]. setting. Similarly, the occasional introduction of new devices In daily practice, there is no external observer present in both the conventional and integrated OR was registered. during a procedure. The only ‘real-time monitoring’ of In this manner, not only the transition to the integrated OR, patient safety is done by the surgeon and/or the entire sur- but also the introduction of new devices was analyzed. All gical team itself. However, from a psychological perspec- procedures were performed by either of the two gynecolo- tive it is known that an individuals’ situational awareness gists with more than 10 years of experience in advanced is impaired when occupied with a (difficult) task [17]. gynecologic laparoscopy and were assisted by one gynecolo- Regarding this phenomenon, implementing new surgical gist who conducted a fellowship in MIS; a group of five devices and technologies in the OR puts more pressure on alternated in the position of either circulating or scrub nurse. the responsibility of the surgeon to maintain surgical safety The study was approved by the Executive Board of the during the whole procedure [1, 15]. The only measures to Haaglanden Medical Center. Prior to the start of the study, enhance safety throughout a procedure that currently are—or all OR personnel were collectively informed about the study. at least should be—used, are the preoperative team briefings, From each patient, informed consent was obtained. This the postoperative debriefings and, to a lesser extent, some design was adapted from another study [3]. preoperative checklists. In general, these safety instruments have proven to diminish preventable errors during the pro- Development of Surgical Safety Questionnaire cedure and to safeguard open communication [18–21]. How- ever, since these tools do not incorporate items to evaluate Patient safety risk factors that have been described by Rod- new surgical techniques or technologies, they are insufficient rigues et al., were summarized in a questionnaire consist- in detecting safety hazards during their introduction. ing of 13 questions (i.e., time-out/sign-out, preparation and Therefore, the aim of this study was to observe whether functioning of devices and instruments, functioning of the surgical teams are capable of measuring surgical safety, surgical team, distracting stimuli, and interference of the especially with regard to the introduction of new techniques study on the procedure) [6]. Directly after each procedure and technologies during a series of MIS procedures. A the (assisting-)surgeon, scrub nurse, and anesthetist(-assis- questionnaire that had to be filled out directly postoperative tant) filled out this short questionnaire. Answers were given was developed to measure surgical safety. Next, the validity on a 5-point Likert scale, ranging from (strongly) disagree of the questionnaire was assessed by comparison with the to (strongly) agree. A potential safety concern was defined results from independent video analysis of these procedures. as an answer between (strongly) disagree and indifferent by at least one member of the surgical team. Additionally, several questions regarding experience (with the procedure, Materials and methods laparoscopy in general, and the used instruments/devices) and the procedure (adhesions, adverse events) were stated In a university-affiliated teaching hospital (Haaglanden Med- (see Online Appendix). ical Center, The Hague), a prospective registration study was set-up to record and analyze surgical flow disturbances. Video analysis During a consecutive series of laparoscopic hysterectomies (LH), a questionnaire was filled out in the OR by the surgical The input from three video signals (endoscopic image and team members. The surgical flow disturbances were scored two dome cameras) and four audio signals (MPEG Recorder by an independent observer. To minimize the interference 2.1) was synchronously recorded during all procedures. The of the study on its own results (the ‘Hawthorne effect’), this recordings were started just before the time-out procedure 1 3 Surgical Endoscopy (2018) 32:3087–3095 3089 and stopped after suturing all port-sites. The procedure was Statistics excluded from analysis in case of technical problems related to the recording equipment. Two residents in Obstetrics and To facilitate statistical analysis, the recordings were Gynecology (M.D.B. and S.R.C.D.) analyzed the presence annotated with The Observer® XT 11.5 software (Nol- and effect of predefined surgical flow disturbances. These dus Information Technologies, Wageningen, The Nether- surgical flow disturbances were defined as stimuli distract- lands). To assess the interobserver variability, a random ing one or more members of the surgical team (Table 1). To sample of six recordings was scored by both observers. assess the severity, the effect of the surgical flow disturbance The findings of the two observers for these six procedures on the surgical team members was graded according to a were compared and the interobserver agreement was cal- seven-point scale. This scale ranges from 1 as a potentially culated (compares events between two observations and distracting stimulus to 7 when the sterile team’s work is takes the frequency and sequence into account; function completely interrupted (modified by Persoon et al. originally incorporated in The Observer® XT 11.5 software). After described by Healey et al.) (Table 2) [9, 22]. satisfactory interobserver agreement was achieved, the remaining procedures were annotated by either one of the two observers (randomly allocated and analyzed in a non-chronological random order) [23, 24]. For statisti- Table 1 Observed types of surgical flow disturbances cal analysis, SPSS 23 statistical software was used. Intra- class correlation coefficient (ICC) was used to assess the Equipment-/instrument-related inter-rater agreement. A two-way random effects model  Set-up device/connection was used since both the procedures as well as the raters  Intraoperative repositioning are a random sample from a larger pool of procedures  Malfunctioning and raters. We checked for consistency (i.e., raters have  Not present a similar pattern of scores). Outcomes are both average  Sterility measures and single measures. Average measures provide  Other/unclear the reliability of the score being able to separate different Environmental levels of safety, despite differences in individual scoring.  Pager/telephone Single Measures represent the reliability you would get if  Door washing room one rater was used. Values between 0.4 and 0.75 were con-  Radio use sidered to represent “fair to good reliability” and > 0.75 Personnel-related “excellent reliability” [25]. In case the kappa becomes  Communication failure negative (due to low variability and high agreement), the  Irrelevant conversation absolute agreement was described as a percentage [26]. Procedure-related A Pearson Chi-square test was used to compare propor-  Extra coagulation bleeding-site tions and a Mann–Whitney U test was used for continuous  Unexpected adhesions variables (non-normally distributed data). A p < .05 was  Limited vision (condensation/smoke) considered statistically significant.  Adverse event  Conversion to laparotomy Table 2 Effect of observed surgical flow disturbances (according to Persoon et al. [9]) 1 Events with the potential to distract the sterile team 2 Sterile team member momentarily distracted: possible involvement of a single sterile member in an event not related to the primary task, e.g., a short head turn in response to a visual or auditory stimulus 3 Sterile team member engages in distraction: similar distraction in 2, but the sterile member engages with the source of distraction by verbally responding while maintaining primary task activity (multitasking) 4 Sterile team member’s primary task interrupted: a single team member ceases his/her current tasks to engage entirely in the distracting stimu- lus 5 Sterile team momentarily distracted: two or more sterile team members respond to a stimulus with a short head turn, no verbal response 6 Sterile team engage in secondary tasks: two or more team members engage with the source of distraction by verbally responding while main- taining primary task activity 7 Sterile team’s work interrupted—operation flow disrupted: interruption of the current primary task of the sterile team, the operation flow is disrupted 1 3 3090 Surgical Endoscopy (2018) 32:3087–3095 Table 3 Patient and procedure characteristics of analyzed LHs per- Results formed in the Haaglanden Medical Center, The Hague, between Janu- ary 2011 and April 2012 During the study period, 84 LHs were performed of which Overall (N = 40) 40 were eligible for inclusion in two studies [3]. For detailed information on the excluded procedures, see Fig. 1. All pro- Median IQR Min–max cedures were successfully completed and three minor post- Age (years) 48.2 43.9–55.2 operative complications were noted (Tables 3, 4). BMI (kg/m ) 24.9 22.7–27.3 The (assisting-)surgeon answered 95% of all questions Uterine weight (g) 165 97–256 [494 out of total 520 questions (40 procedures, 13 ques- Operating time (min) 121 ± 29 66–176 tions per procedure)], the scrub nurse answered 89% (461 Procedure time (min) 156 ± 31 98–215 out of 520), and the anesthetist(-assistant) answered 86% Estimated blood loss (mL) 100 50–175 of the questions (445 out of 520). Based on the question- Hospital stay (days) 2.0 1.1–2.1 naire, all surgical team members were of the opinion that Benign indication (%) 70.0% the study did not interfere with the procedure in 33 out of the 40 procedures (83%). In all cases, one of the two expe- IQR inter quartile range (25th and 75th percentile), BMI Body Mass Index rienced gynecologists (> 100 LHs) attended the procedure. Time between first incision and last suture (skin-to-skin) (based on Nevertheless, the questionnaire was filled out in 58% of video observation) the cases by the assisting surgeon. As a result, reported Time between patient entering OR and leaving OR (based on video experience of the surgeon with LH varied between ≤ 25 observation) prior procedures (14%), 26–40 (30%), 41–100 (32%), and > 100 prior LHs in 24% of the procedures. The surgeons MIS was in 37% of the cases between 41 and 100 and reported in 41% of the cases to have used the same instru- in 53% > 100 prior procedures. Despite this, experience ments and devices > 100 times before in prior procedures. with LH specifically was moderate; in 71% of the cases In 50% they reported to have experience with the equip- the scrub nurse had performed ≤ 25 prior LH procedures. ment between 25 and 100 prior procedures and in 8% this Similarly, their experience with the equipment was moder- was ≤ 25 procedures. Experience of the scrub nurse with ate (in 43–47% of the cases ≤ 25 procedures). Total performed procedures during study period N=84 Procedures in Procedures in other integrated Procedures in conventional OR OR (during construction) integrated OR N=11 N=27 N=46 Excluded: 4 no informed consent Excluded: 5 problem video recording Excluded: 3 no informed consent 6 problem audio recording 11 not registered 2 technical failure 3 other reasons 2 other reasons First 8 procedures due to maximum of 20 reached Eligible recorded procedures Eligible recorded procedures conventional OR integrated OR N=20 N=20 Total recordings used for analysis N=40 Fig. 1 Inclusion of eligible procedures 1 3 Surgical Endoscopy (2018) 32:3087–3095 3091 Table 4 Adverse events all analyzed LHs In general, scores given by the surgeon were in 16% (85/520) regarded as a potential safety concern. With respect Overall (N = 40) to the scrub nurse and anesthesiologist this was both 9% Infection 1 (2.5%) (46/520). Overall, ‘strongly disagree’ was reported in 2% Blood loss > 1L 1 (2.5%) (9/520), of which 8 were reported on questions 3, 4, or 5 Others 1 (2.5%) (i.e., equipment related, see Online Appendix). Total 3 (7.5%) In 87% (452 of 520 questions), all members of the surgi- cal team agreed in their answers (i.e., the maximum differ - All adverse events did not require re-operation and occurred postop- ence between the lowest and the highest was ≤ one point on eratively the Likert scale). In 4% (22 of 520), the absolute difference LH Laparoscopic hysterectomy between the members of the surgical team was high (≥ 3; for Urinary tract infection example, to the same question the surgeon reports ‘disagree’ Postoperative drop in hemoglobin. CT-scan showed free fluid intra- abdominally. Vital signs were stable and after a blood transfusion and the scrub nurse reports ‘strongly agree’). The ICC was with two packed cells hemoglobin levels remained stable 0.44 (average measures). Patient suffered from sensibility loss in her right hand. The neurolo- gist diagnosed a neurapraxia of the median nerve. Conservative man- Validation of Surgical Safety Questionnaire by video agement resulted in almost complete recovery analysis The overall observation duration of these procedures was Surgical Safety Questionnaire 103 h and 45 min. Six randomly chosen observations were annotated by both observers and showed excellent agree- The scores per question of the individual team members are summarized in Table  5. In 15% (6 out of 40) of the ment (Cohen’s Kappa of 0.79–0.98, all observations com- bined 0.85, p < .001). Therefore, the remaining procedures procedures, potential safety concerns [i.e., answer ‘indif- ferent’ or ‘(strongly) disagree’] were reported regarding were annotated by the two observers separately (in total 36 observations by M.D.B. and 10 by S.R.C.D., respectively). the time-out and sign-out procedure. With respect to the preparation, functioning, and ease of use of the devices The duration and effect of disturbances during procedures in which a potential safety concern was reported with regard in 37.5–47.5% (15–19 out of 40 procedures) a potential safety concern was reported by one or more team mem- to the functioning of devices and instruments (question 4, see Online Appendix) were compared to the procedures in bers. A strong disagreement to a flawless use of the devices was reported in seven procedures (17.5%). With respect to which no safety concern was reported (Table 6). In the pro- cedures after which a potential safety concern was reported, communication and collaboration in 30–35% (12–14 out of 40 procedures) concerns were reported, mostly by the a significantly higher percentage of the duration of the pro- cedure consisted of surgical flow disturbances [9.3 ± 6.2 vs. surgeon. Table 5 scores per question of the team members individually Question Surgeon Scrub nurse Anesthetist N Mean ± SD Range N ≤ 3 N Mean ± SD Range N ≤ 3 N Mean ± SD Range N ≤ 3 Time-out 39 4.54 ± 0.55 3–5 1 36 4.19 ± 0.67 2–5 3 37 4.08 ± 0.68 2–5 3 Sign-out 37 4.49 ± 0.51 4–5 0 31 4.16 ± 0.86 2–5 5 28 3.96 ± 0.51 2–5 2 Preparation 39 3.97 ± 1.06 1–5 11 36 4.14 ± 0.72 2–5 5 34 3.88 ± 0.81 2–5 7 Functioning 39 3.51 ± 1.21 1–5 16 36 3.83 ± 1.11 1–5 6 33 3.85 ± 0.67 2–5 6 Ease of use 39 3.82 ± 1.07 1–5 11 36 3.94 ± 0.83 2–5 5 32 3.81 ± 0.74 1–5 7 Communication 39 3.9 ± 0.75 2–5 11 35 3.86 ± 0.77 2–5 5 36 4.11 ± 0.52 3–5 3 Collaboration 39 3.92 ± 0.74 2–5 10 36 3.89 ± 0.62 2–5 5 36 4.14 ± 0.42 3–5 1 Disturbances 39 3.95 ± 0.92 2–5 7 36 3.89 ± 0.85 1–5 4 35 3.77 ± 0.81 2–5 8 Surgeon 28 3.96 ± 0.43 3–5 3 36 4.25 ± 0.55 3–5 2 35 4.14 ± 0.49 3–5 2 Scrub nurse 39 3.92 ± 0.62 2–5 7 35 4 ± 0.48 3–5 4 35 4.14 ± 0.43 3–5 1 Anesthetist 39 4.18 ± 0.51 3–5 2 36 4.19 ± 0.47 3–5 1 32 4.41 ± 0.5 4–5 0 Patient safety 39 4.21 ± 0.7 3–5 4 36 4.08 ± 0.5 2–5 1 36 4.42 ± 0.5 4–5 0 Study influence 39 4.56 ± 0.6 3–5 2 36 4.31 ± 0.47 4–5 0 36 3.97 ± 0.81 2–5 6 N ≤ 3: The number of questions to which a score ≤ 3 was given, which is defined as a safety concern 1 3 3092 Surgical Endoscopy (2018) 32:3087–3095 Table 6 Duration and effect of surgical flow disturbances with regard to functioning of devices and instruments (question 4 of questionnaire) separated between procedures with or without a safety concern reported by at least one member of the surgical team (N = 40 procedures) d d No safety concern reported Safety concern reported N Mean ± SD Min–max N Mean ± SD Min–max p Percentage of procedure 21 2.9 ± 3.7 0.0–15.4 19 9.3 ± 6.2 1.6–21.7 < .001 Effect (weighted) 21 4.4 ± 2.4 0.0–7.0 19 6.1 ± 1.9 3.0–7.0 .020 Impact 21 13.2 ± 12.0 0.0–47.1 19 56.2 ± 38.7 11.5–145.7 < .001 SD standard deviation Total duration of the disturbance (s) defined as percentage of the total procedure time Effect of the disturbance (based on Persoon et al. [9]) corrected by the duration of the disturbance(s) Percentage of procedure multiplied by weighted effect Reported answer by at least one surgical team member was (strongly) disagree or indifferent Mann–Whitney U test for independent samples 2.9 ± 3.7% (mean ± SD), p < .001]. Similarly, in these proce- suture for the vaginal cu ff (one procedure), and multiple new dures, a significantly higher mean weighted effect (i.e., the devices/instruments (three procedures). mean effect of the disturbances corrected for the duration of the disturbances) was found (score 6.1 ± 1.9 vs. 4.4 ± 2.4, Experience p = .020; see Table 2 for the meaning of the scores). In the group without any reported safety concerns, there Limited experience of the scrub nurse with the equipment were only two procedures during which a relatively high (≤ 25 procedures) resulted in significantly more potential percentage of the procedure consisted of disturbances (10.0 safety concerns reported by at least one member of the sur- and 15.4%, respectively). However, the mean weighted effect gical team (30.7 vs. 15.6%, p = .002). However, this did not of these disturbances was low (1.9 and 3.0, respectively) and result in a higher percentage of procedure time expended to therefore can be regarded as adequately managed. All tests to surgical flow disturbances (7.3 ± 7.6 vs. 5.0 ± 5.2%, p = .423) assess whether using the questionnaire of one or two of the and/or a higher effect of these disturbances (5.7 ± 1.4 vs. team members might be applicable as well resulted in lower 4.8 ± 2.3, p = .275) (N = 30 procedures). Experience of the agreement with the video analysis (not shown). surgeon with the used instruments did not have a significant influence on the potential safety concerns either (25.6 vs. Newly introduced devices and/or technology 23.8%, p = .791). During eight procedures (20%, four procedures in the con- ventional OR and four in the integrated OR), a new instru- Discussion ment and/or device was used. During these procedures, the surgical team members reported a potential safety concern in The Surgical Safety Questionnaire filled out directly post - 51% (41 out of 80 questions regarding intraoperative aspects operative by all members of the surgical team proved to be (question 3 till 12), see Online Appendix). In contrast, the a valid tool to adequately estimate surgical safety in MIS. prevalence of a potential safety concern during the other Procedures during which a relatively high percentage of the procedures was 23.1% (74 out of 320, p < .001). duration consisted of surgical flow disturbances and/or with The first 20 procedures were performed in a conventional a relatively high mean weighted effect of these disturbances cart-based OR. The last 20 procedures were performed in a matched with the reported potential safety concerns. Fur- new integrated OR. No difference in potential safety con- thermore, during procedures in which a new instrument or cerns was reported between the two OR set-ups (28 vs. 29%, device was used, significantly more potential safety concerns p = .740). Furthermore, an employee of the medical industry were reported by the surgical team. Therefore this could be was present during seven procedures (four in conventional a useful tool in the evaluation and maintenance of surgical OR, three in integrated OR), during which a newly intro- safety during the introduction of new surgical equipment or duced device was used. Additionally, in one procedure a technology. new device was used without an employee of the industry The term patient safety is at risk to become an empty being present (fourth consecutive procedure in which this phrase by its broad interpretation. To define nuances in instrument was used). The new equipment concerned a new patient safety, the ‘systems approach’ is most commonly bipolar sealing instrument (five procedures), a new type of used [27, 28]. Based on this approach, several studies 1 3 Surgical Endoscopy (2018) 32:3087–3095 3093 introduced frameworks covering the risk domains relevant Furthermore, in contrast to the high agreement (87%), the to surgical safety and patient outcomes [6, 7, 29]. The reported ICC (0.44) seems low. However, this discrepancy questionnaire validated in present study covers these risk is explained by the low variability and high agreement in domains and thereby provides a composite outcome for sur- the reported answers. In those cases, kappa is not a reliable gical safety. estimate for correlation [26]. Thirdly, the reported experi- A study conducted by Russ et al. had similar objectives ence with the LH seems low. This is due to the system in and described the Metric for Evaluating Task Execution in The Netherlands, in which residents specializing in MIS the Operating Room (METEOR) as an easy to use tool to are usually allowed to perform LH as ‘primary’ surgeon allow surgical teams to self-assess their performance, in during the last year of their residency and therefore also order to track surgical hazards, and to be able to evaluate filled out our scoring sheets. However, without exception, safety [30]. However, their checklist is quite extensive (up in these cases, the senior consultant with extensive expe- to 80 items) and does not cover concerns regarding instru- rience in advanced gynecologic endoscopy was always ments and devices. Since the high dependency on technol- member of the sterile team as well. ogy in MIS, equipment-related disturbances are one of the Over the past decades patient outcomes regarding MIS well-known primary sources of disruption [3, 8, 31]. Addi- have rapidly improved. Large leaps could be made in the tionally, during the introduction of a new technique and/or early days of MIS, where measures taken to improve safety technology in the OR, disruptions are even more likely to were highly effective. Currently, only smaller steps can be occur [4, 7]. This hazard is also one of the main results in made with a higher risk of doing harm instead of good [1, our study. Therefore, prior to the introduction of a new inter- 35]. Furthermore, the OR has become increasingly com- vention in the OR, a prospective risk analysis should be per- plex. As Sir Cyril Chantler said: “Medicine used to be formed to guarantee safe implementation (e.g., Healthcare simple, ineffective and relatively safe. Now it is complex, Failure Mode and Effect Analysis) [32]. Nevertheless, in effective and potentially dangerous” [36]. The common our opinion, methods currently used to monitor this imple- objective we are pursuing is to enable technology to assist mentation (i.e., evaluation after 6 and 12 months, adverse the surgeon and its team in maintaining surgical safety. events registration, incident reporting system) fail to detect Similar to recent developments in the automotive industry safety concerns in a timely manner. Similarly, our results to assist the driver on traffic safety (e.g., collision avoid- rule out the widespread assumption that an employee of the ance, blind spot detection, and lane departure warning medical industry being present can prevent safety hazards. systems), some promising systems are currently tested in Instead, the Surgical Safety Questionnaire presented in this a few hospitals in The Netherlands. For example, the Digi- study could be a useful tool to systematically evaluate the tal Operating Room Assistant continuously monitors the surgical safety after each procedure, especially in case of the location, status, and (mal)functioning of devices [37, 38]. introduction of a new instrument or technology. In conclusion, the results of our study demonstrate that The main strength of our study is that by using video the presented Surgical Safety Questionnaire can act as a observation we were able to assess surgical flow distur - validated tool to evaluate and maintain surgical safety dur- bances without influencing the course of the procedure. ing minimally invasive procedures. In daily practice, we In that way, we obtained very reliable quantitative results recommend to fill out this questionnaire in case a new to serve as gold standard and thereby allowing validation technique or technology is used during a procedure. By of our Surgical Safety Questionnaire. This is in line with involving the complete surgical team with their individual other studies recognizing the additional value of detailed knowledge, experience, and opinions, this will provide analysis of video registration [33, 34]. A weakness could the opportunity to constantly evaluate new equipment and be that scoring on a 5-point Likert scale remains prone techniques. As a consequence, in an early stage, potential to subjectivity. What determines the difference between safety hazards will be prevented in future patients. agree, neither agree nor disagree, and disagreement? It Acknowledgements The authors want to acknowledge Arjan van Dijke was decided to place the cut-off for a potential safety con - for his extensive help with the video observation system. cern at ‘neither agree nor disagree.’ By doing so, every time at least one of the team members for any reason had Compliance with ethical standards a motive to not (fully) agree on a certain question in the questionnaire, the item was marked as potential safety con- Disclosures Mathijs Blikkendaal, Sara Driessen, Sharon Rodrigues, cern. Nevertheless, the results of our study indicate that by Johann Rhemrev, Maddy Smeets, Jenny Dankelman, John Dobbelsteen, and Frank Willem Jansen have no conflicts of interest or financial ties using this definition the potential safety concerns correlate to disclose. very well with the observed surgical flow disturbances. 1 3 3094 Surgical Endoscopy (2018) 32:3087–3095 Open Access This article is distributed under the terms of the Creative 17. Johnson HL, Kimsey D (2012) Patient safety: break the silence. Commons Attribution 4.0 International License (http://creativecom- AORN J 95:591–601 mons.org/licenses/by/4.0/), which permits unrestricted use, distribu- 18. Treadwell JR, Lucas S, Tsou AY (2014) Surgical checklists: a tion, and reproduction in any medium, provided you give appropriate systematic review of impacts and implementation. BMJ Qual Saf credit to the original author(s) and the source, provide a link to the 23:299–318 Creative Commons license, and indicate if changes were made. 19. Gluck PA (2012) Patient safety: some progress and many chal- lenges. Obstet Gynecol 120:1149–1159 20. de Vries EN, Prins HA, Crolla RM, den Outer AJ, van Andel G, van Helden SH, Schlack WS, van Putten MA, Gouma DJ, Dijk- References graaf MG, Smorenburg SM, Boermeester MA (2010) Effect of a comprehensive surgical safety system on patient outcomes. New 1. Eisenberg D, Wren SM (2014) Innovation in safety, and safety in Engl J Med 363:1928–1937 innovation. JAMA Surg 149:7–9 21. Birkmeyer JD (2012) Progress and challenges in improving surgi- 2. Sevdalis N, Hull L, Birnbach DJ (2012) Improving patient safety cal outcomes. Br J Surg 99:1467–1469 in the operating theatre and perioperative care: obstacles, inter- 22. Healey AN, Primus CP, Koutantji M (2007) Quantifying distrac- ventions, and priorities for accelerating progress. Br J Anaesth tion and interruption in urological surgery. Qual Saf Health Care 109(Suppl 1):i3–i16 16:135–139 3. Blikkendaal MD, Driessen SR, Rodrigues SP, Rhemrev JP, Smeets 23. Buzink SN, van LL, de Hingh, Jakimowicz IH JJ (2010) Risk- MJ, Dankelman J, van den Dobbelsteen JJ, Jansen FW (2017) Sur- sensitive events during laparoscopic cholecystectomy: the influ- gical flow disturbances in dedicated minimally invasive surgery ence of the integrated operating room and a preoperative checklist suites: an observational study to assess its supposed superiority tool. Surg Endosc 24:1990–1995 over conventional suites. Surg Endosc 31:288–298 24. Zheng B, Martinec DV, Cassera MA, Swanstrom LL (2008) A 4. Antoniadis S, Passauer-Baierl S, Baschnegger H, Weigl M (2014) quantitative study of disruption in the operating room during lapa- Identification and interference of intraoperative distractions and roscopic antireflux surgery. Surg Endosc 22:2171–2177 interruptions in operating rooms. J Surg Res 188:21–29 25. Fleiss JL (1986) The design and analysis of clinical experiments. 5. Bonrath EM, Dedy NJ, Zevin B, Grantcharov TP (2013) Defining Wiley, New York technical errors in laparoscopic surgery: a systematic review. Surg 26. Feinstein AR, Cicchetti DV (1990) High agreement but low kappa: Endosc 27:2678–2691 I. The problems of two paradoxes. J Clin Epidemiol 43:543–549 6. Rodrigues SP, Wever AM, Dankelman J, Jansen FW (2012) Risk 27. Vincent C, Moorthy K, Sarker SK, Chang A, Darzi AW (2004) factors in patient safety: minimally invasive surgery versus con- Systems approaches to surgical quality and safety: from concept ventional surgery. Surg Endosc 26:350–356 to measurement. Ann Surg 239:475–482 7. Driessen SRC, Sandberg EM, Rodrigues SP, van Zwet EW, Jansen 28. Dankelman J, Grimbergen CA (2005) Systems approach to reduce FW (2017) Identification of risk factors in minimally invasive sur - errors in surgery. Surg Endosc 19:1017–1021 gery: a prospective multicenter study. Surg Endosc 31:2467–2473 29. Calland JF, Guerlain S, Adams RB, Tribble CG, Foley E, Chekan 8. Weerakkody RA, Cheshire NJ, Riga C, Lear R, Hamady MS, EG (2002) A systems approach to surgical safety. Surg Endosc Moorthy K, Darzi AW, Vincent C, Bicknell CD (2013) Surgi- 16:1005–1014 cal technology and operating-room safety failures: a systematic 30. Russ S, Arora S, Wharton R, Wheelock A, Hull L, Sharma E, review of quantitative studies. BMJ Qual Saf 22:710–718 Darzi A, Vincent C, Sevdalis N (2013) Measuring safety and effi- 9. Persoon MC, Broos HJ, Witjes JA, Hendrikx AJ, Scherpbier AJ ciency in the operating room: development and validation of a (2011) The effect of distractions in the operating room during metric for evaluating task execution in the operating room. J Am endourological procedures. Surg Endosc 25:437–443 Coll Surg 216:472–481 10. Parker SE, Laviana AA, Wadhera RK, Wiegmann DA, Sundt TM 31. Verdaasdonk EG, Stassen LP, van der Elst M, Karsten TM, III (2010) Development and evaluation of an observational tool Dankelman J (2007) Problems with technical equipment dur- for assessing surgical flow disruptions and their impact on surgical ing laparoscopic surgery. An observational study. Surg Endosc performance. World J Surg 34:353–361 21:275–279 11. Pronovost PJ, Thompson DA, Holzmueller CG, Lubomski LH, 32. Linkin DR, Sausman C, Santos L, Lyons C, Fox C, Aumiller Morlock LL (2005) Defining and measuring patient safety. Crit L, Esterhai J, Pittman B, Lautenbach E (2005) Applicability of Care Clin 21:1–19 healthcare failure mode and effects analysis to healthcare epide- 12. Reason J (2004) Beyond the organisational accident: the need for miology: evaluation of the sterilization and use of surgical instru- “error wisdom” on the frontline. Qual Saf Health Care 13(Suppl ments. Clin Infect Dis 41:1014–1019 2):ii28–ii33 33. Bezemer J, Cope A, Korkiakangas T, Kress G, Murtagh G, Wel- 13. Wiegmann DA, ElBardissi AW, Dearani JA, Daly RC, Sundt TM don SM, Kneebone R (2017) Microanalysis of video from the III (2007) Disruptions in surgical flow and their relationship to operating room: an underused approach to patient safety research. surgical errors: an exploratory investigation. Surgery 142:658–665 BMJ Qual Saf 26:583–587 14. Weigl M, Antoniadis S, Chiapponi C, Bruns C, Sevdalis N (2015) 34. Makary MA (2013) The power of video recording: taking quality The impact of intra-operative interruptions on surgeons’ perceived to the next level. JAMA 309:1591–1592 workload: an observational study in elective general and orthope- 35. Parsons JK, Messer K, Palazzi K, Stroup SP, Chang D (2014) dic surgery. Surg Endosc 29:145–153 Diffusion of surgical innovations, patient safety, and minimally 15. Pluyter JR, Buzink SN, Rutkowski AF, Jakimowicz JJ (2010) Do invasive radical prostatectomy. JAMA Surg 149:845–851 absorption and realistic distraction influence performance of com- 36. Chantler C (1999) The role and education of doctors in the deliv- ponent task surgical procedure? Surg Endosc 24:902–907 ery of health care. Lancet 353:1178–1181 16. Hsu KE, Man FY, Gizicki RA, Feldman LS, Fried GM (2008) 37. Guedon AC, Wauben LS, Overvelde M, Blok JH, van der Elst M, Experienced surgeons can do more than one thing at a time: Dankelman J, van den Dobbelsteen JJ (2014) Safety status system effect of distraction on performance of a simple laparoscopic and for operating room devices. Technol Health Care 22:795–803 cognitive task by experienced and novice surgeons. Surg Endosc 22:196–201 1 3 Surgical Endoscopy (2018) 32:3087–3095 3095 38. Henken KR, Jansen FW, Klein J, Stassen LP, Dankelman J, van recording in clinical practice. Surg Endosc 26:2909–2916 den Dobbelsteen JJ (2012) Implications of the law on video 1 3

Journal

Surgical EndoscopySpringer Journals

Published: Jan 19, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off