Wide variation and patterns of physicians’ responses to drug–drug interaction alerts

Wide variation and patterns of physicians’ responses to drug–drug interaction alerts Abstract Objectives Providing physicians with alerts about potentially harmful drug–drug interactions (DDIs) is only moderately effective due to high alert override rates. To understand high override behavior on DDI alerts, we investigated how physicians respond to DDIs and their behavior patterns and variations. Design Retrospective system log data analysis and records review (sampling 2% of total overrides). Setting A large tertiary academic hospital. Participants About 560 physicians and their override responses to DDI alerts generated from 1 September to 31 December 2014. Interventions Not applicable. Main Outcome Measure(s) DDI alert frequency and override rate. Results We found significant variation in both the number of alerts and override rates at the levels of physicians, departments and drug-class pairs. Physician-level variations were wider for residents than for faculty staff (number of alerts: t = 254.17, P = 0.011; override rates: t = –4.77, P < 0.0001). Using the number of alerts and their override rate, we classified physicians into four groups: inexperienced incautious users, inexperienced cautious users, experienced cautious users and experienced incautious users. Medical department influenced both alert numbers and override rates. Nearly 90% of the overrides involved only five drug-class combinations, which had a wide range of appropriateness in the chart review. Conclusion The variations at drug-class levels suggest issues with system design and the DDI rules. Department-level variation may be best addressed at the department level, and the rest of the variation appears related to individual physician responses, suggesting the need for interventions at an individual level. computerized physician order entry, drug–drug interaction, alert override, behavior pattern, variation analysis Introduction Drug–drug interaction (DDI) alerts represent a widely used form of clinical decision support for preventing medication errors, and they are considered to have great promise for improving medication safety. Despite the potential benefits of such alerts, physicians frequently decide to ignore them and override alerts without modifying prescriptions. These override rates are extremely high even for high-severity alerts. Addressing this issue requires the specificity of alerts to be improved, which might be achievable in several ways, including focusing on high-priority DDIs [1, 2], tiering DDIs according to their severity [3], considering patient factors and co-medications in order to suppress insignificant alerts [4], using sophisticated algorithms that dynamically monitor changes in patient parameters over time [5, 6] or using a combination of these approaches. Despite these efforts, alert override rates remain stubbornly high and also inconsistent across institutions [7, 8]. The terms ‘alert fatigue’ and ‘alert flood’ describe a state in which physicians become desensitized in the setting of frequent, low-specificity alerts and these terms are often used to explain high override rates [9]. This state is frequently associated with a large number of alerts or alert overload, and so it can be addressed by decreasing the alert burden [10, 11]. A recent study [10] proposed that alert fatigue develops over time during increased exposure to alerts, leading to frequent override behavior. The concept of alert fatigue is based on the relationship between the override rate and the number of alerts experienced, but details around exactly how this relationship works such as how many unnecessary alerts it takes to produce alert fatigue remain unclear. One study that explored the relationships among the numbers of prescriptions, alerts experienced and overrides in an outpatient setting found no correlation between the number of alerts and the override rate [12] and this was also the case in another study [8]. The override rate has traditionally been viewed as a surrogate inverse indicator of alert effectiveness. The reported override rates on medication alerts have varied widely (from 24% to 98%) with the alert type and study site, and these rates are usually higher in outpatient settings than in inpatient settings [13–15]. Several studies have addressed the variation in the responses of physicians to alerts vary with the DDI content and alert system [9, 12, 16]. Variation in the prescribing behaviors of physicians are widespread in healthcare and often result in lower levels of safety and high levels of waste [17]. Based on US national data, Zhang et al. [17] found that lower-quality prescribing patterns were associated with both higher costs and higher rates of adverse drug events. However, there have been few investigations of the effects of physician-level variations in DDI alerts. The objective of this study was to identify variations in the use of DDI alerts at the physician level to obtain information that can be used to ameliorate high override rates. We focused on (i) determining the prescribing patterns of individual physicians and their responses to DDI alerts, (ii) identifying the prescribing patterns of physicians with high override rates and how they differ from those with low override rates and (iii) investigating the clinical appropriateness of overrides through a chart-review process. Methods Study site and setting We evaluated all inpatient departments of a tertiary teaching hospital in Seoul, Republic of Korea with ~2700 beds and 912 300 admissions annually. This inpatient setting provided 63 specialty services covering general medical, surgical, pediatric and oncology units, and 7 intensive-care units. There were 1156 physicians affiliated to the hospital, excluding the departments of laboratory medicine, pathology, nuclear medicine, clinical pharmacology and radiology. These physicians included 472 residents, who are usually those responsible for inpatient prescriptions. The hospital had used a computerized physician order entry (CPOE) system with a locally developed complete electronic medical record (EMR) system for more than 10 years. The system was used throughout the hospital in both the inpatient and outpatient settings. DDI-checking alerts were instituted as a decision-support component of the CPOE system for the following five types of alerts based on the prospective Drug Utilization Review program, which has been a government standard since 2012: DDI, pregnancy checking, age checking, drug duplication and formulary. The DDI rules involve 706 drug pairs with 77 drug-class combinations. At inpatient setting, patient orders were prescribed daily basis. When physicians receive a DDI alert, they can cancel the order they are writing or discontinue a preexisting drug order, or they can continue the order and choose the reason for the override from several coded options. Study design and data selection This study had a cross-sectional, observational design and involved DDI alerts generated from 1 September to 31 December 2014. We received approval from the institutional review board of the hospital to access the DDI alert data (IRB # 2014-1101). The 4-month data collection period was determined based on the number of physicians as an analysis unit, as follows: referring to our previous study [12] of how the characteristics of physicians affect alerts, an effective sample size of 376 physicians was required to achieve a statistical power of 80% in detecting minimum clinically significant differences in the mean levels of the alerts, overrides and override rate between physicians with high and low override rates under two-sided testing with a type-I error probability (alpha value) of 5%. In anticipation that variance inflation would occur, in particular due to the inclusion of physicians with no or very few DDI alerts triggered and from the possibility of up to 50% of physician information not being available, we planned to obtain alert logs from 560 physicians so that the required sample size was achieved. This number of physicians could be included if data were collected for 4 months. The DDI log of the CPOE system revealed that 378 physicians had received at least 1 alert. For each alert, we collected the encoded identifier of the patient, the names of the two drugs, the identifier of the physician who ordered the drugs, the date of the alert, the response of the physician to the alert and the coded reason(s) for the override as entered by the physician. The following information about each physician was retrieved separately from the hospital administrative information system in June 2015: demographics (age, gender and physician type), educational background (medical school, residency training and specialty), medical department and mean number of monthly prescriptions. Electronic medical record review To examine the clinical context and appropriateness of alerts that were frequently overridden, two physicians conducted a detailed chart review independently for 253 randomly selected overrides, which constituted about 2% of the total number of overrides. The chart-review process for assessing appropriate overrides was set up as shown in Fig. 1. We used a stepwise approach considering multiple data in a stepwise manner. First, we examined the objective and precipitate drugs if one of them is topically applied, ophthalmic and otic preparations to be appropriate. Alert overrides of DDIs involving an epinephrine autoinjector were also considered appropriate. Second, we reviewed EMRs of demographics, medical diagnosis, history of disease, problem and medication list, physician notes and laboratory results to obtain information about what clinical event happened to a patient around the prescription date. Third, we reviewed the override reason entered by a physician to determine whether it was a ‘prn’, whether it was prescribed in different times, or whether no substitute drugs under the given condition. The interrater agreement coefficient determined from independent reviews of the first 50 overrides was 0.74, which improved to 0.92 in the second-round review of an additional 50 overrides. Any disagreements were resolved by discussion. Figure 1 View largeDownload slide Chart review process for determining the appropriateness of overrides. Figure 1 View largeDownload slide Chart review process for determining the appropriateness of overrides. Data analysis There was a total of 18 360 alerts for prescriptions ordered by 378 physicians, of which 13 155 were overridden, giving an override rate of 71.7%. Profile data were missing for 29 physicians, and so the data of 349 physicians were analyzed further. We plotted the relationship between alert frequency and override rate to identify physicians who might be experiencing alert fatigue. We moved the reference coordinate to 18 alerts and used an override rate of 70% [referred to below as a reference coordinate of (18, 70%)]—based on the median number of alerts and the mean override rate—to group the physicians according to their behavior types. The effect of physician characteristics on the override rate was explored using the chi-square test, ANOVA and generalized linear regression. The effect of patient characteristics on number of alerts and override rates were examined through linear mixed effect model using Poisson and binominal distribution. Descriptive statistics were used to summarize the combination of drug classes responsible for generating most of the alerts and alert overrides. To explore the overriding patterns of physicians by drug-class combination, we compared the overrides of physicians ranked in the highest versus lowest override-rate quintiles. The statistical analyses were performed using SAS software (version 9.3, SAS Institute, Cary, NC, USA). Results Physician-level variation The demographics information indicated that 57% (n = 197) of the physicians were male, and they were aged 32.19 ± 5.80 years (mean ± SD). One-quarter of the physicians were faculty staff (n = 89) with a medical specialty, and three-quarters (n = 260) of them were residents. Among the physician characteristics, the physician type significantly affected the number of alerts (t = 254.17, P = 0.01) and override rate (t = −4.77, P < 0.01): the number of alerts was higher and had a wider variance for residents than for faculty staff, while the override rate was lower for residents than for faculty staff but had a similar variance. There were no differences in analysis of age, sex and educational background variables. No relationship was found between the number of alerts and the override rate among physicians (r = 0.04, P = 0.69). The effect of patient characteristics was no significant (intra-class correlation <0.0001). Using the reference coordinates (18, 70%), physicians were grouped into the following four quadrants (Fig. 2): Q1, inexperienced incautious users, with high alert frequencies and high override rates; Q2, inexperienced cautious users, with high alert frequencies and low override rates; Q3, experienced cautious users, with low alert frequencies and low override rates and Q4, experienced incautious users, with low alert frequencies and high override rates. Faculty staff and residents tended to be concentrated in quadrants Q4 and Q2, respectively. Figure 2 View largeDownload slide Grouping of physicians based on the median number of alerts triggered and mean override rates. n indicates the number of physicians; the percentage within parentheses is the proportion of residents. Figure 2 View largeDownload slide Grouping of physicians based on the median number of alerts triggered and mean override rates. n indicates the number of physicians; the percentage within parentheses is the proportion of residents. Department-level variation Figure 3 shows the patterns of relationships between alerts and overrides for four groups of medical departments. The responses for chest surgery (CS) were concentrated around the mean override rate, with a long tail. The responses for the departments of anesthesiology and pain medicine (APM), family medicine (FM), obstetrics and gynecology (OBGY) and neurosurgery (NS) were concentrated on the left of the graph with high override rates. In contrast, most of the responses for the department of internal medicine (IM) were below the reference line with a smaller number of alerts, while those of the other departments were scattered randomly. There were significant differences in the patterns for the numbers of alerts (F = 38.34, P < 0.01) and override rates (F = 20.39, P < 0.01). The other departments included surgery other than CS (n = 79) and small numbers of physicians had received alerts in psychiatrics, urology, emergency, ophthalmology and neurology departments. Five psychiatric physicians overrode all the alerts they received, while the override rates were below the reference line for physicians in urology (n = 8) and emergency (n = 11) departments. Physicians in ophthalmology (n = 8) and neurology (n = 7) departments overrode around 70% of the alerts they received. Figure 3 View largeDownload slide Shows the relationships between the number of alerts and override rate by department. The dotted line indicates the reference coordinate (18, 70%). APM, anesthesiology and pain medicine; CS, chest surgery; FM, family medicine; IM, internal medicine; NS, neurosurgery; OBGY, obstetrics and gynecology; Other departments include surgery other than CS, and psychiatrics, urology, emergency, ophthalmology and neurology departments. Figure 3 View largeDownload slide Shows the relationships between the number of alerts and override rate by department. The dotted line indicates the reference coordinate (18, 70%). APM, anesthesiology and pain medicine; CS, chest surgery; FM, family medicine; IM, internal medicine; NS, neurosurgery; OBGY, obstetrics and gynecology; Other departments include surgery other than CS, and psychiatrics, urology, emergency, ophthalmology and neurology departments. Drug-class-level variations and override appropriateness Five drug-class combinations out of 77 class pairs comprised 89.4% of all overrides (Table 1). These drug combinations corresponded to 28 drug pairs. The chart review of override appropriateness revealed that the combination of QT-prolonging agents and beta-adrenergic blockers or amphetamine derivatives, and the combination of non-steroidal anti-inflammatory drugs (NSAIDs) and aspirin salicylates showed high appropriateness values of 75.3% and 66.7%, respectively. Combinations of NSAIDs and combinations of potassium-sparing diuretics and potassium preparations showed low appropriateness due to physicians entering inappropriate reasons for overriding alerts. Table 1 All overrides, and override appropriateness in the chart-review sample Object class  Precipitate class  Overall overrides (n = 13 155)  Chart review sample  No. of overrides  Appropriate overrides  Appropriate override rate (%)  QT-prolonging agents  Beta-adrenergic blockers/amphetamine and derivatives  3949 (30.0%)  93 (36.6%)  70  75.3  NSAIDs  NSAIDs  3738 (28.4%)  83 (32.8%)  12  14.5  Potassium-sparing diuretics  Potassium preparations  3085 (23.5%)  34 (13.4%)  16  47.1  NSAIDs  Aspirin salicylates  986 (7.5%)  27 (10.7%)  18  66.7  Subtotal  11 758 (89.4%)  237 (93.7%)  116  48.9  Other combinations  1397 (10.6%)  16 (6.3%)  4  25.0  Total  13 155 (100%)  253 (100%)  120  47.4  Object class  Precipitate class  Overall overrides (n = 13 155)  Chart review sample  No. of overrides  Appropriate overrides  Appropriate override rate (%)  QT-prolonging agents  Beta-adrenergic blockers/amphetamine and derivatives  3949 (30.0%)  93 (36.6%)  70  75.3  NSAIDs  NSAIDs  3738 (28.4%)  83 (32.8%)  12  14.5  Potassium-sparing diuretics  Potassium preparations  3085 (23.5%)  34 (13.4%)  16  47.1  NSAIDs  Aspirin salicylates  986 (7.5%)  27 (10.7%)  18  66.7  Subtotal  11 758 (89.4%)  237 (93.7%)  116  48.9  Other combinations  1397 (10.6%)  16 (6.3%)  4  25.0  Total  13 155 (100%)  253 (100%)  120  47.4  Data are n (%) values. NSAIDs, non-steroidal anti-inflammatory drugs. Table 1 All overrides, and override appropriateness in the chart-review sample Object class  Precipitate class  Overall overrides (n = 13 155)  Chart review sample  No. of overrides  Appropriate overrides  Appropriate override rate (%)  QT-prolonging agents  Beta-adrenergic blockers/amphetamine and derivatives  3949 (30.0%)  93 (36.6%)  70  75.3  NSAIDs  NSAIDs  3738 (28.4%)  83 (32.8%)  12  14.5  Potassium-sparing diuretics  Potassium preparations  3085 (23.5%)  34 (13.4%)  16  47.1  NSAIDs  Aspirin salicylates  986 (7.5%)  27 (10.7%)  18  66.7  Subtotal  11 758 (89.4%)  237 (93.7%)  116  48.9  Other combinations  1397 (10.6%)  16 (6.3%)  4  25.0  Total  13 155 (100%)  253 (100%)  120  47.4  Object class  Precipitate class  Overall overrides (n = 13 155)  Chart review sample  No. of overrides  Appropriate overrides  Appropriate override rate (%)  QT-prolonging agents  Beta-adrenergic blockers/amphetamine and derivatives  3949 (30.0%)  93 (36.6%)  70  75.3  NSAIDs  NSAIDs  3738 (28.4%)  83 (32.8%)  12  14.5  Potassium-sparing diuretics  Potassium preparations  3085 (23.5%)  34 (13.4%)  16  47.1  NSAIDs  Aspirin salicylates  986 (7.5%)  27 (10.7%)  18  66.7  Subtotal  11 758 (89.4%)  237 (93.7%)  116  48.9  Other combinations  1397 (10.6%)  16 (6.3%)  4  25.0  Total  13 155 (100%)  253 (100%)  120  47.4  Data are n (%) values. NSAIDs, non-steroidal anti-inflammatory drugs. A comparison of the highest versus lowest override-rate quintile groups revealed that physicians in the highest quintile overrode 96% alerts of QT-prolonging agents and amphetamine derivatives (Table 2). Meanwhile, in the lowest quintile this combination was prescribed rarely, and only half the alerts were overridden. Combinations of NSAIDs and combinations of NSAIDs with beta-adrenergic blockers or aspirin salicylates had high override rates in the highest quintile; these combinations were rarely prescribed in the lowest quintile. The lowest quintile was characterized by high alert frequencies and a high override rate only for the combination of potassium-sparing diuretics and preparations. Table 2 Comparison of alert frequencies and override rates by drug-class combination between the highest and lowest quintile physicians Object class  Precipitate class  Highest quintile (n = 46)  Lowest quintile (n = 46)  QT-prolonging agents  Amphetamine and derivatives  994 (96.4%)  37 (51.9%)  QT-prolonging agents  Beta-adrenergic blockers  111 (96.0%)  457 (46.6%)  NSAIDs  NSAIDs  761 (86.8%)  42 (70.2%)  NSAIDs  Beta-adrenergic blockers  197 (89.1%)  0  Potassium-sparing diuretics  Potassium-sparing/preparation  21 (95.5%)  235 (75.6%)  NSAIDs  Aspirin salicylates  140 (76.9%)  17 (58.2%)  Potassium-sparing diuretics  Macrolide immunosuppressives  32 (100.0%)  70 (60.8%)  Antibiotics  Macrolide immunosuppressives  0  19 (28.5%)  QT-prolonging agents  QT-prolonging agents  46 (95.8%)  8 (33.3%)  Object class  Precipitate class  Highest quintile (n = 46)  Lowest quintile (n = 46)  QT-prolonging agents  Amphetamine and derivatives  994 (96.4%)  37 (51.9%)  QT-prolonging agents  Beta-adrenergic blockers  111 (96.0%)  457 (46.6%)  NSAIDs  NSAIDs  761 (86.8%)  42 (70.2%)  NSAIDs  Beta-adrenergic blockers  197 (89.1%)  0  Potassium-sparing diuretics  Potassium-sparing/preparation  21 (95.5%)  235 (75.6%)  NSAIDs  Aspirin salicylates  140 (76.9%)  17 (58.2%)  Potassium-sparing diuretics  Macrolide immunosuppressives  32 (100.0%)  70 (60.8%)  Antibiotics  Macrolide immunosuppressives  0  19 (28.5%)  QT-prolonging agents  QT-prolonging agents  46 (95.8%)  8 (33.3%)  Data are number of alerts (percentage override rate) values. Table 2 Comparison of alert frequencies and override rates by drug-class combination between the highest and lowest quintile physicians Object class  Precipitate class  Highest quintile (n = 46)  Lowest quintile (n = 46)  QT-prolonging agents  Amphetamine and derivatives  994 (96.4%)  37 (51.9%)  QT-prolonging agents  Beta-adrenergic blockers  111 (96.0%)  457 (46.6%)  NSAIDs  NSAIDs  761 (86.8%)  42 (70.2%)  NSAIDs  Beta-adrenergic blockers  197 (89.1%)  0  Potassium-sparing diuretics  Potassium-sparing/preparation  21 (95.5%)  235 (75.6%)  NSAIDs  Aspirin salicylates  140 (76.9%)  17 (58.2%)  Potassium-sparing diuretics  Macrolide immunosuppressives  32 (100.0%)  70 (60.8%)  Antibiotics  Macrolide immunosuppressives  0  19 (28.5%)  QT-prolonging agents  QT-prolonging agents  46 (95.8%)  8 (33.3%)  Object class  Precipitate class  Highest quintile (n = 46)  Lowest quintile (n = 46)  QT-prolonging agents  Amphetamine and derivatives  994 (96.4%)  37 (51.9%)  QT-prolonging agents  Beta-adrenergic blockers  111 (96.0%)  457 (46.6%)  NSAIDs  NSAIDs  761 (86.8%)  42 (70.2%)  NSAIDs  Beta-adrenergic blockers  197 (89.1%)  0  Potassium-sparing diuretics  Potassium-sparing/preparation  21 (95.5%)  235 (75.6%)  NSAIDs  Aspirin salicylates  140 (76.9%)  17 (58.2%)  Potassium-sparing diuretics  Macrolide immunosuppressives  32 (100.0%)  70 (60.8%)  Antibiotics  Macrolide immunosuppressives  0  19 (28.5%)  QT-prolonging agents  QT-prolonging agents  46 (95.8%)  8 (33.3%)  Data are number of alerts (percentage override rate) values. Discussion We examined the DDI override behaviors of physicians using physician profiles and CPOE logs in the inpatient setting, and found that a small number of physicians triggered most of the alerts, and their override rates varied widely. The physician-level variations were larger for residents than for faculty or staff. These data suggest that targeting specific physicians might be an effective intervention. The highly skewed distribution of alerts was consistent with a previous study [8] which also found that a small number of physicians triggered a high proportion of alerts with widely varying override rates. However, the other study did not explore this result further, whereas we examined the variation at three levels. For physician level, the residents in quadrant Q1 showed different response patterns with those in Q2 and Q3. It is not clear why they behaved, but we could assume either (i) they prescribe drugs in very different clinical contexts, we call it as perceptual override or (ii) they repeated override automatically without conscious intention as habitual override [10]. The physicians in quadrants Q2 and Q3 (inexperienced and experienced cautious users) benefitted the most from the DDI alert system. The behaviors in Q4 likely do not represent alert fatigue or flood. Regarding department-level variation, each of the four department groups showed typical patterns. The CS department triggered large numbers of alerts on potassium-combination prescriptions after cardiac surgeries that were almost routinely overridden. In our chart review, the potassium-combination prescriptions were frequently accompanied with daily monitoring of serum potassium levels, which were considered reasonable. However, the override reasons that physicians entered were ‘prn’, ‘different time’ or ‘no replacement’, which were judged as being inappropriate. The APM, FM, OBGY and NS departments made the largest contributions to the high overall override rate. The APM physicians had two main kinds of work: pain management and anesthesia during surgical operation. Combinations of QT-prolonging agents and beta-adrenergic blockers or amphetamine derivatives were commonly unaccepted by anesthesiologists, who closely monitor patients using devices throughout surgical operations. The main alerts in FM were NSAIDs–aspirin combinations, while the main overrides in OBGY and NS were NSAID–NSAID combinations to control acute pain, followed by surgical procedures. Intervening in only these combinations could have decreased the frequency of overrides more than 50%. Another particularly interesting finding was that only five drug-class combinations were responsible for almost 90% of all overrides. A study conducted in Switzerland similarly found that four to ten drug pairs accounted for 52–97% of severe overrides [18]. A study conducted in the Netherlands [14] found that 10 and 50 drug-class combinations contributed to >50% and >90% of all DDI alerts, respectively. A common feature of these studies is that the DDI rules were based on national standards that are used in all hospitals. Zwart-van Rijkom et al. [14] explained that their findings were related to the national DDI standard primarily being developed for use in community pharmacies. In our chart review, a large portion of overrides were related to the risks of hyperkalemia and gastrointestinal bleeding (among the five drug-class combinations) which were used to prevent a decrease in serum potassium after cardiac surgery or for postoperative pain control. Hypokalemia is known as a strong independent predictor of mortality in patients with heart failure. Potassium replacement therapy is desirable to avoid hypokalemia proactively by the aggressive replacement of potassium [19]. As for postoperative pain control, use of opioids can be alternatives of NSAID combinations that are associated with a risk of gastrointestinal bleeding. However, postoperative opioid-related respiratory depression is reportedly an important cause of iatrogenic injury in the perioperative period [20]. Therefore, for the inpatient setting those rules should be modified by relevant variables to increase rule specificity which could decrease override rates by at least 50%. Combining the results of the appropriateness chart review and comparing the highest- and lowest-quintile physicians yielded clues for which rules should be modified or conditionally constrained. In addition, some interventions will likely need to target providers with high override rates. For example, for combinations of QT-prolonging agents–beta-adrenergic blockers or amphetamine derivatives which are not used in anesthesia, the most effective interventions might be behavioral ones designed to appeal to provider self-image and social motivations. A study [21] reported that the use of accountable justification and peer comparison as behavioral interventions resulted in lower rates of inappropriate antibiotic prescribing for acute respiratory tract infections. The accountable justification was to prompt physicians to enter a free-text justification for prescribing antibiotics on the EMRs, so that other providers could see it. If he or she did not enter a justification, the statement ‘No justification for prescribing antibiotics was given’ would be entered into the EMR. The present results must be interpreted within the context of the study design. This study has two limits. First, it was conducted in a single center and focused on one type of medication alert of DDI, so our results might not be generalizable to healthcare systems such as community health systems, subacute hospitals or other tertiary hospitals with different DDI rule sets of government or commercial products in other countries. Second, this study has a cross-sectional design, so it is hard to establish a causal relationship. For the external validity, future studies are needed with attempting to identify how such habituation occurs over time and who is vulnerable to it. The strength of this study is that we investigated physicians’ behavior patterns and variation, which is the lack of studies in this field. Conclusions We found a high override rate on DDI alerts and wide variation at the levels of physicians, medical departments and drug classes. These variations are interrelated and provide clues about which requirements are not met by the DDI alert system and which rules may have a low specificity in the clinical context of different medical departments. Two kinds of physician response on alerts contributed to the wide variation: habitual and perceptual overrides. Efforts to improve the acceptance rate of DDIs should be initiated and designed based on a systematic understanding of the behaviors and patterns physicians exhibit in accepting and over-riding decision support. Funding This research was supported by a Korea Research Foundation Grant, which was funded by the Korean Government (MOEHRD; No. NRF-2016R1D1A1A09919502). D.B. was sponsored by the Centers for Education and Research on Therapeutics (CERT grant number 1U19HS021094-01), Agency for Healthcare Research and Quality, Rockville, MD, USA. References 1 McEvoy DS, Sittig DF, Hickman T-T et al.  . Variation in high-priority drug–drug interaction alerts across institutions and electronic health records. J Am Med Inform Assoc  2016; 24: 331– 8. 2 Phansalkar S, Desai AA, Bell D et al.  . High-priority drug–drug interactions for use in electronic health records. J Am Med Inform Assoc  2012; 19: 735– 43. Google Scholar CrossRef Search ADS PubMed  3 Paterno MD, Maviglia SM, Gorman PN et al.  . Tiering drug–drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc  2009; 16: 40– 6. Google Scholar CrossRef Search ADS PubMed  4 Seidling HM, Klein U, Schaier M et al.  . What, if all alerts were specific–estimating the potential impact on drug interaction alert burden. Int J Med Inform  2014; 83: 285– 91. Google Scholar CrossRef Search ADS PubMed  5 Eschmann E, Beeler PE, Schneemann M et al.  . Developing strategies for predicting hyperkalemia in potassium-increasing drug–drug interactions. J Am Med Inform Assoc  2017; 24: 60– 6. Google Scholar CrossRef Search ADS PubMed  6 Duke JD, Li X, Dexter P. Adherence to drug—drug interaction alerts in high-risk patients: a trial of context-enhanced alerting. J Am Med Inform Assoc  2013; 20: 494– 8. Google Scholar CrossRef Search ADS PubMed  7 Schreiber R, Gregoire JA, Shaha JE et al.  . Think time: a novel approach to analysis of clinicians’ behavior after reduction of drug–drug interaction alerts. Int J Med Inform  2017; 97: 59– 67. Google Scholar CrossRef Search ADS PubMed  8 Bryant A, Fletcher G, Payne T. Drug interaction alert override rates in the meaningful use era: no evidence of progress. Appl Clin Inform  2014; 5: 802– 13. Google Scholar CrossRef Search ADS PubMed  9 Slight SP, Seger DL, Nanji KC et al.  . Are we heeding the warning signs? Examining providers’ overrides of computerized drug–drug interaction alerts in primary care. PLoS ONE  2013; 8: e85071. Google Scholar CrossRef Search ADS PubMed  10 Baysari MT, Tariq A, Day RO et al.  . Alert override as a habitual behavior—a new perspective on a persistent problem. J Am Med Inform Assoc  2017; 24: 409– 12. Google Scholar PubMed  11 Tamblyn R, Reidel K, Patel V. Physicians’ response to computerised alerts for psychotropic drugs in older persons: a multilevel analysis of the associated alert, patient and physician characteristics. BMJ Open  2012; 2: e001384. Google Scholar CrossRef Search ADS PubMed  12 Cho I, Slight SP, Nanji KC et al.  . The effect of provider characteristics on the responses to medication-related decision support alerts. Int J Med Inform  2015; 84: 630– 9. Google Scholar CrossRef Search ADS PubMed  13 Nanji KC, Slight SP, Seger DL et al.  . Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc  2014; 21: 487– 91. Google Scholar CrossRef Search ADS PubMed  14 Zwart-van Rijkom JEF, Uijtendaal EV, Ten Berg MJ et al.  . Frequency and nature of drug–drug interactions in a Dutch university hospital. Br J Clin Pharmacol  2009; 68: 187– 93. Google Scholar CrossRef Search ADS PubMed  15 Simpao AF, Ahumada LM, Desai BR et al.  . Optimization of drug–drug interaction alert rules in a pediatric hospital’s electronic health record system using a visual analytics dashboard. J Am Med Inform Assoc  2015; 22: 361– 9. Google Scholar PubMed  16 Cho I, Slight SP, Nanji KC et al.  . Understanding physicians’ behavior toward alerts about nephrotoxic medications in outpatients: a cross-sectional analysis. BMC Nephrol  2014; 15: 200. Google Scholar CrossRef Search ADS PubMed  17 Zhang Y, Baicker K, Newhouse JP. Geographic variation in the quality of prescribing. N Engl J Med  2010; 363: 1985– 8. Google Scholar CrossRef Search ADS PubMed  18 Oertle M. Frequency and nature of drug–drug interactions in a Swiss primary and secondary acute care hospital. Swiss Med Wkly  2012; 142: w13522. Google Scholar PubMed  19 Merchant Q, Rehman Siddiqui NU, Rehmat A et al.  . Comparison of Enteral versus Intravenous Potassium Supplementation in hypokalaemia in postcardiac surgery paediatric cardiac intensive care patients: prospective open label randomised control trial (EIPS). BMJ Open  2014; 4: e005124. Google Scholar CrossRef Search ADS PubMed  20 Lee LA, Caplan RA, Stephens LS et al.  . Postoperative opioid-induced respiratory depression: a closed claims analysis. Anesthesiology  2015; 122: 659– 65. Google Scholar CrossRef Search ADS PubMed  21 Meeker D, Linder JA, Fox CR et al.  . Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial. JAMA  2016; 315: 562– 70. Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press in association with the International Society for Quality in Health Care. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png International Journal for Quality in Health Care Oxford University Press

Wide variation and patterns of physicians’ responses to drug–drug interaction alerts

Loading next page...
 
/lp/ou_press/wide-variation-and-patterns-of-physicians-responses-to-drug-drug-oHhlywtACB
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press in association with the International Society for Quality in Health Care. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com
ISSN
1353-4505
eISSN
1464-3677
D.O.I.
10.1093/intqhc/mzy102
Publisher site
See Article on Publisher Site

Abstract

Abstract Objectives Providing physicians with alerts about potentially harmful drug–drug interactions (DDIs) is only moderately effective due to high alert override rates. To understand high override behavior on DDI alerts, we investigated how physicians respond to DDIs and their behavior patterns and variations. Design Retrospective system log data analysis and records review (sampling 2% of total overrides). Setting A large tertiary academic hospital. Participants About 560 physicians and their override responses to DDI alerts generated from 1 September to 31 December 2014. Interventions Not applicable. Main Outcome Measure(s) DDI alert frequency and override rate. Results We found significant variation in both the number of alerts and override rates at the levels of physicians, departments and drug-class pairs. Physician-level variations were wider for residents than for faculty staff (number of alerts: t = 254.17, P = 0.011; override rates: t = –4.77, P < 0.0001). Using the number of alerts and their override rate, we classified physicians into four groups: inexperienced incautious users, inexperienced cautious users, experienced cautious users and experienced incautious users. Medical department influenced both alert numbers and override rates. Nearly 90% of the overrides involved only five drug-class combinations, which had a wide range of appropriateness in the chart review. Conclusion The variations at drug-class levels suggest issues with system design and the DDI rules. Department-level variation may be best addressed at the department level, and the rest of the variation appears related to individual physician responses, suggesting the need for interventions at an individual level. computerized physician order entry, drug–drug interaction, alert override, behavior pattern, variation analysis Introduction Drug–drug interaction (DDI) alerts represent a widely used form of clinical decision support for preventing medication errors, and they are considered to have great promise for improving medication safety. Despite the potential benefits of such alerts, physicians frequently decide to ignore them and override alerts without modifying prescriptions. These override rates are extremely high even for high-severity alerts. Addressing this issue requires the specificity of alerts to be improved, which might be achievable in several ways, including focusing on high-priority DDIs [1, 2], tiering DDIs according to their severity [3], considering patient factors and co-medications in order to suppress insignificant alerts [4], using sophisticated algorithms that dynamically monitor changes in patient parameters over time [5, 6] or using a combination of these approaches. Despite these efforts, alert override rates remain stubbornly high and also inconsistent across institutions [7, 8]. The terms ‘alert fatigue’ and ‘alert flood’ describe a state in which physicians become desensitized in the setting of frequent, low-specificity alerts and these terms are often used to explain high override rates [9]. This state is frequently associated with a large number of alerts or alert overload, and so it can be addressed by decreasing the alert burden [10, 11]. A recent study [10] proposed that alert fatigue develops over time during increased exposure to alerts, leading to frequent override behavior. The concept of alert fatigue is based on the relationship between the override rate and the number of alerts experienced, but details around exactly how this relationship works such as how many unnecessary alerts it takes to produce alert fatigue remain unclear. One study that explored the relationships among the numbers of prescriptions, alerts experienced and overrides in an outpatient setting found no correlation between the number of alerts and the override rate [12] and this was also the case in another study [8]. The override rate has traditionally been viewed as a surrogate inverse indicator of alert effectiveness. The reported override rates on medication alerts have varied widely (from 24% to 98%) with the alert type and study site, and these rates are usually higher in outpatient settings than in inpatient settings [13–15]. Several studies have addressed the variation in the responses of physicians to alerts vary with the DDI content and alert system [9, 12, 16]. Variation in the prescribing behaviors of physicians are widespread in healthcare and often result in lower levels of safety and high levels of waste [17]. Based on US national data, Zhang et al. [17] found that lower-quality prescribing patterns were associated with both higher costs and higher rates of adverse drug events. However, there have been few investigations of the effects of physician-level variations in DDI alerts. The objective of this study was to identify variations in the use of DDI alerts at the physician level to obtain information that can be used to ameliorate high override rates. We focused on (i) determining the prescribing patterns of individual physicians and their responses to DDI alerts, (ii) identifying the prescribing patterns of physicians with high override rates and how they differ from those with low override rates and (iii) investigating the clinical appropriateness of overrides through a chart-review process. Methods Study site and setting We evaluated all inpatient departments of a tertiary teaching hospital in Seoul, Republic of Korea with ~2700 beds and 912 300 admissions annually. This inpatient setting provided 63 specialty services covering general medical, surgical, pediatric and oncology units, and 7 intensive-care units. There were 1156 physicians affiliated to the hospital, excluding the departments of laboratory medicine, pathology, nuclear medicine, clinical pharmacology and radiology. These physicians included 472 residents, who are usually those responsible for inpatient prescriptions. The hospital had used a computerized physician order entry (CPOE) system with a locally developed complete electronic medical record (EMR) system for more than 10 years. The system was used throughout the hospital in both the inpatient and outpatient settings. DDI-checking alerts were instituted as a decision-support component of the CPOE system for the following five types of alerts based on the prospective Drug Utilization Review program, which has been a government standard since 2012: DDI, pregnancy checking, age checking, drug duplication and formulary. The DDI rules involve 706 drug pairs with 77 drug-class combinations. At inpatient setting, patient orders were prescribed daily basis. When physicians receive a DDI alert, they can cancel the order they are writing or discontinue a preexisting drug order, or they can continue the order and choose the reason for the override from several coded options. Study design and data selection This study had a cross-sectional, observational design and involved DDI alerts generated from 1 September to 31 December 2014. We received approval from the institutional review board of the hospital to access the DDI alert data (IRB # 2014-1101). The 4-month data collection period was determined based on the number of physicians as an analysis unit, as follows: referring to our previous study [12] of how the characteristics of physicians affect alerts, an effective sample size of 376 physicians was required to achieve a statistical power of 80% in detecting minimum clinically significant differences in the mean levels of the alerts, overrides and override rate between physicians with high and low override rates under two-sided testing with a type-I error probability (alpha value) of 5%. In anticipation that variance inflation would occur, in particular due to the inclusion of physicians with no or very few DDI alerts triggered and from the possibility of up to 50% of physician information not being available, we planned to obtain alert logs from 560 physicians so that the required sample size was achieved. This number of physicians could be included if data were collected for 4 months. The DDI log of the CPOE system revealed that 378 physicians had received at least 1 alert. For each alert, we collected the encoded identifier of the patient, the names of the two drugs, the identifier of the physician who ordered the drugs, the date of the alert, the response of the physician to the alert and the coded reason(s) for the override as entered by the physician. The following information about each physician was retrieved separately from the hospital administrative information system in June 2015: demographics (age, gender and physician type), educational background (medical school, residency training and specialty), medical department and mean number of monthly prescriptions. Electronic medical record review To examine the clinical context and appropriateness of alerts that were frequently overridden, two physicians conducted a detailed chart review independently for 253 randomly selected overrides, which constituted about 2% of the total number of overrides. The chart-review process for assessing appropriate overrides was set up as shown in Fig. 1. We used a stepwise approach considering multiple data in a stepwise manner. First, we examined the objective and precipitate drugs if one of them is topically applied, ophthalmic and otic preparations to be appropriate. Alert overrides of DDIs involving an epinephrine autoinjector were also considered appropriate. Second, we reviewed EMRs of demographics, medical diagnosis, history of disease, problem and medication list, physician notes and laboratory results to obtain information about what clinical event happened to a patient around the prescription date. Third, we reviewed the override reason entered by a physician to determine whether it was a ‘prn’, whether it was prescribed in different times, or whether no substitute drugs under the given condition. The interrater agreement coefficient determined from independent reviews of the first 50 overrides was 0.74, which improved to 0.92 in the second-round review of an additional 50 overrides. Any disagreements were resolved by discussion. Figure 1 View largeDownload slide Chart review process for determining the appropriateness of overrides. Figure 1 View largeDownload slide Chart review process for determining the appropriateness of overrides. Data analysis There was a total of 18 360 alerts for prescriptions ordered by 378 physicians, of which 13 155 were overridden, giving an override rate of 71.7%. Profile data were missing for 29 physicians, and so the data of 349 physicians were analyzed further. We plotted the relationship between alert frequency and override rate to identify physicians who might be experiencing alert fatigue. We moved the reference coordinate to 18 alerts and used an override rate of 70% [referred to below as a reference coordinate of (18, 70%)]—based on the median number of alerts and the mean override rate—to group the physicians according to their behavior types. The effect of physician characteristics on the override rate was explored using the chi-square test, ANOVA and generalized linear regression. The effect of patient characteristics on number of alerts and override rates were examined through linear mixed effect model using Poisson and binominal distribution. Descriptive statistics were used to summarize the combination of drug classes responsible for generating most of the alerts and alert overrides. To explore the overriding patterns of physicians by drug-class combination, we compared the overrides of physicians ranked in the highest versus lowest override-rate quintiles. The statistical analyses were performed using SAS software (version 9.3, SAS Institute, Cary, NC, USA). Results Physician-level variation The demographics information indicated that 57% (n = 197) of the physicians were male, and they were aged 32.19 ± 5.80 years (mean ± SD). One-quarter of the physicians were faculty staff (n = 89) with a medical specialty, and three-quarters (n = 260) of them were residents. Among the physician characteristics, the physician type significantly affected the number of alerts (t = 254.17, P = 0.01) and override rate (t = −4.77, P < 0.01): the number of alerts was higher and had a wider variance for residents than for faculty staff, while the override rate was lower for residents than for faculty staff but had a similar variance. There were no differences in analysis of age, sex and educational background variables. No relationship was found between the number of alerts and the override rate among physicians (r = 0.04, P = 0.69). The effect of patient characteristics was no significant (intra-class correlation <0.0001). Using the reference coordinates (18, 70%), physicians were grouped into the following four quadrants (Fig. 2): Q1, inexperienced incautious users, with high alert frequencies and high override rates; Q2, inexperienced cautious users, with high alert frequencies and low override rates; Q3, experienced cautious users, with low alert frequencies and low override rates and Q4, experienced incautious users, with low alert frequencies and high override rates. Faculty staff and residents tended to be concentrated in quadrants Q4 and Q2, respectively. Figure 2 View largeDownload slide Grouping of physicians based on the median number of alerts triggered and mean override rates. n indicates the number of physicians; the percentage within parentheses is the proportion of residents. Figure 2 View largeDownload slide Grouping of physicians based on the median number of alerts triggered and mean override rates. n indicates the number of physicians; the percentage within parentheses is the proportion of residents. Department-level variation Figure 3 shows the patterns of relationships between alerts and overrides for four groups of medical departments. The responses for chest surgery (CS) were concentrated around the mean override rate, with a long tail. The responses for the departments of anesthesiology and pain medicine (APM), family medicine (FM), obstetrics and gynecology (OBGY) and neurosurgery (NS) were concentrated on the left of the graph with high override rates. In contrast, most of the responses for the department of internal medicine (IM) were below the reference line with a smaller number of alerts, while those of the other departments were scattered randomly. There were significant differences in the patterns for the numbers of alerts (F = 38.34, P < 0.01) and override rates (F = 20.39, P < 0.01). The other departments included surgery other than CS (n = 79) and small numbers of physicians had received alerts in psychiatrics, urology, emergency, ophthalmology and neurology departments. Five psychiatric physicians overrode all the alerts they received, while the override rates were below the reference line for physicians in urology (n = 8) and emergency (n = 11) departments. Physicians in ophthalmology (n = 8) and neurology (n = 7) departments overrode around 70% of the alerts they received. Figure 3 View largeDownload slide Shows the relationships between the number of alerts and override rate by department. The dotted line indicates the reference coordinate (18, 70%). APM, anesthesiology and pain medicine; CS, chest surgery; FM, family medicine; IM, internal medicine; NS, neurosurgery; OBGY, obstetrics and gynecology; Other departments include surgery other than CS, and psychiatrics, urology, emergency, ophthalmology and neurology departments. Figure 3 View largeDownload slide Shows the relationships between the number of alerts and override rate by department. The dotted line indicates the reference coordinate (18, 70%). APM, anesthesiology and pain medicine; CS, chest surgery; FM, family medicine; IM, internal medicine; NS, neurosurgery; OBGY, obstetrics and gynecology; Other departments include surgery other than CS, and psychiatrics, urology, emergency, ophthalmology and neurology departments. Drug-class-level variations and override appropriateness Five drug-class combinations out of 77 class pairs comprised 89.4% of all overrides (Table 1). These drug combinations corresponded to 28 drug pairs. The chart review of override appropriateness revealed that the combination of QT-prolonging agents and beta-adrenergic blockers or amphetamine derivatives, and the combination of non-steroidal anti-inflammatory drugs (NSAIDs) and aspirin salicylates showed high appropriateness values of 75.3% and 66.7%, respectively. Combinations of NSAIDs and combinations of potassium-sparing diuretics and potassium preparations showed low appropriateness due to physicians entering inappropriate reasons for overriding alerts. Table 1 All overrides, and override appropriateness in the chart-review sample Object class  Precipitate class  Overall overrides (n = 13 155)  Chart review sample  No. of overrides  Appropriate overrides  Appropriate override rate (%)  QT-prolonging agents  Beta-adrenergic blockers/amphetamine and derivatives  3949 (30.0%)  93 (36.6%)  70  75.3  NSAIDs  NSAIDs  3738 (28.4%)  83 (32.8%)  12  14.5  Potassium-sparing diuretics  Potassium preparations  3085 (23.5%)  34 (13.4%)  16  47.1  NSAIDs  Aspirin salicylates  986 (7.5%)  27 (10.7%)  18  66.7  Subtotal  11 758 (89.4%)  237 (93.7%)  116  48.9  Other combinations  1397 (10.6%)  16 (6.3%)  4  25.0  Total  13 155 (100%)  253 (100%)  120  47.4  Object class  Precipitate class  Overall overrides (n = 13 155)  Chart review sample  No. of overrides  Appropriate overrides  Appropriate override rate (%)  QT-prolonging agents  Beta-adrenergic blockers/amphetamine and derivatives  3949 (30.0%)  93 (36.6%)  70  75.3  NSAIDs  NSAIDs  3738 (28.4%)  83 (32.8%)  12  14.5  Potassium-sparing diuretics  Potassium preparations  3085 (23.5%)  34 (13.4%)  16  47.1  NSAIDs  Aspirin salicylates  986 (7.5%)  27 (10.7%)  18  66.7  Subtotal  11 758 (89.4%)  237 (93.7%)  116  48.9  Other combinations  1397 (10.6%)  16 (6.3%)  4  25.0  Total  13 155 (100%)  253 (100%)  120  47.4  Data are n (%) values. NSAIDs, non-steroidal anti-inflammatory drugs. Table 1 All overrides, and override appropriateness in the chart-review sample Object class  Precipitate class  Overall overrides (n = 13 155)  Chart review sample  No. of overrides  Appropriate overrides  Appropriate override rate (%)  QT-prolonging agents  Beta-adrenergic blockers/amphetamine and derivatives  3949 (30.0%)  93 (36.6%)  70  75.3  NSAIDs  NSAIDs  3738 (28.4%)  83 (32.8%)  12  14.5  Potassium-sparing diuretics  Potassium preparations  3085 (23.5%)  34 (13.4%)  16  47.1  NSAIDs  Aspirin salicylates  986 (7.5%)  27 (10.7%)  18  66.7  Subtotal  11 758 (89.4%)  237 (93.7%)  116  48.9  Other combinations  1397 (10.6%)  16 (6.3%)  4  25.0  Total  13 155 (100%)  253 (100%)  120  47.4  Object class  Precipitate class  Overall overrides (n = 13 155)  Chart review sample  No. of overrides  Appropriate overrides  Appropriate override rate (%)  QT-prolonging agents  Beta-adrenergic blockers/amphetamine and derivatives  3949 (30.0%)  93 (36.6%)  70  75.3  NSAIDs  NSAIDs  3738 (28.4%)  83 (32.8%)  12  14.5  Potassium-sparing diuretics  Potassium preparations  3085 (23.5%)  34 (13.4%)  16  47.1  NSAIDs  Aspirin salicylates  986 (7.5%)  27 (10.7%)  18  66.7  Subtotal  11 758 (89.4%)  237 (93.7%)  116  48.9  Other combinations  1397 (10.6%)  16 (6.3%)  4  25.0  Total  13 155 (100%)  253 (100%)  120  47.4  Data are n (%) values. NSAIDs, non-steroidal anti-inflammatory drugs. A comparison of the highest versus lowest override-rate quintile groups revealed that physicians in the highest quintile overrode 96% alerts of QT-prolonging agents and amphetamine derivatives (Table 2). Meanwhile, in the lowest quintile this combination was prescribed rarely, and only half the alerts were overridden. Combinations of NSAIDs and combinations of NSAIDs with beta-adrenergic blockers or aspirin salicylates had high override rates in the highest quintile; these combinations were rarely prescribed in the lowest quintile. The lowest quintile was characterized by high alert frequencies and a high override rate only for the combination of potassium-sparing diuretics and preparations. Table 2 Comparison of alert frequencies and override rates by drug-class combination between the highest and lowest quintile physicians Object class  Precipitate class  Highest quintile (n = 46)  Lowest quintile (n = 46)  QT-prolonging agents  Amphetamine and derivatives  994 (96.4%)  37 (51.9%)  QT-prolonging agents  Beta-adrenergic blockers  111 (96.0%)  457 (46.6%)  NSAIDs  NSAIDs  761 (86.8%)  42 (70.2%)  NSAIDs  Beta-adrenergic blockers  197 (89.1%)  0  Potassium-sparing diuretics  Potassium-sparing/preparation  21 (95.5%)  235 (75.6%)  NSAIDs  Aspirin salicylates  140 (76.9%)  17 (58.2%)  Potassium-sparing diuretics  Macrolide immunosuppressives  32 (100.0%)  70 (60.8%)  Antibiotics  Macrolide immunosuppressives  0  19 (28.5%)  QT-prolonging agents  QT-prolonging agents  46 (95.8%)  8 (33.3%)  Object class  Precipitate class  Highest quintile (n = 46)  Lowest quintile (n = 46)  QT-prolonging agents  Amphetamine and derivatives  994 (96.4%)  37 (51.9%)  QT-prolonging agents  Beta-adrenergic blockers  111 (96.0%)  457 (46.6%)  NSAIDs  NSAIDs  761 (86.8%)  42 (70.2%)  NSAIDs  Beta-adrenergic blockers  197 (89.1%)  0  Potassium-sparing diuretics  Potassium-sparing/preparation  21 (95.5%)  235 (75.6%)  NSAIDs  Aspirin salicylates  140 (76.9%)  17 (58.2%)  Potassium-sparing diuretics  Macrolide immunosuppressives  32 (100.0%)  70 (60.8%)  Antibiotics  Macrolide immunosuppressives  0  19 (28.5%)  QT-prolonging agents  QT-prolonging agents  46 (95.8%)  8 (33.3%)  Data are number of alerts (percentage override rate) values. Table 2 Comparison of alert frequencies and override rates by drug-class combination between the highest and lowest quintile physicians Object class  Precipitate class  Highest quintile (n = 46)  Lowest quintile (n = 46)  QT-prolonging agents  Amphetamine and derivatives  994 (96.4%)  37 (51.9%)  QT-prolonging agents  Beta-adrenergic blockers  111 (96.0%)  457 (46.6%)  NSAIDs  NSAIDs  761 (86.8%)  42 (70.2%)  NSAIDs  Beta-adrenergic blockers  197 (89.1%)  0  Potassium-sparing diuretics  Potassium-sparing/preparation  21 (95.5%)  235 (75.6%)  NSAIDs  Aspirin salicylates  140 (76.9%)  17 (58.2%)  Potassium-sparing diuretics  Macrolide immunosuppressives  32 (100.0%)  70 (60.8%)  Antibiotics  Macrolide immunosuppressives  0  19 (28.5%)  QT-prolonging agents  QT-prolonging agents  46 (95.8%)  8 (33.3%)  Object class  Precipitate class  Highest quintile (n = 46)  Lowest quintile (n = 46)  QT-prolonging agents  Amphetamine and derivatives  994 (96.4%)  37 (51.9%)  QT-prolonging agents  Beta-adrenergic blockers  111 (96.0%)  457 (46.6%)  NSAIDs  NSAIDs  761 (86.8%)  42 (70.2%)  NSAIDs  Beta-adrenergic blockers  197 (89.1%)  0  Potassium-sparing diuretics  Potassium-sparing/preparation  21 (95.5%)  235 (75.6%)  NSAIDs  Aspirin salicylates  140 (76.9%)  17 (58.2%)  Potassium-sparing diuretics  Macrolide immunosuppressives  32 (100.0%)  70 (60.8%)  Antibiotics  Macrolide immunosuppressives  0  19 (28.5%)  QT-prolonging agents  QT-prolonging agents  46 (95.8%)  8 (33.3%)  Data are number of alerts (percentage override rate) values. Discussion We examined the DDI override behaviors of physicians using physician profiles and CPOE logs in the inpatient setting, and found that a small number of physicians triggered most of the alerts, and their override rates varied widely. The physician-level variations were larger for residents than for faculty or staff. These data suggest that targeting specific physicians might be an effective intervention. The highly skewed distribution of alerts was consistent with a previous study [8] which also found that a small number of physicians triggered a high proportion of alerts with widely varying override rates. However, the other study did not explore this result further, whereas we examined the variation at three levels. For physician level, the residents in quadrant Q1 showed different response patterns with those in Q2 and Q3. It is not clear why they behaved, but we could assume either (i) they prescribe drugs in very different clinical contexts, we call it as perceptual override or (ii) they repeated override automatically without conscious intention as habitual override [10]. The physicians in quadrants Q2 and Q3 (inexperienced and experienced cautious users) benefitted the most from the DDI alert system. The behaviors in Q4 likely do not represent alert fatigue or flood. Regarding department-level variation, each of the four department groups showed typical patterns. The CS department triggered large numbers of alerts on potassium-combination prescriptions after cardiac surgeries that were almost routinely overridden. In our chart review, the potassium-combination prescriptions were frequently accompanied with daily monitoring of serum potassium levels, which were considered reasonable. However, the override reasons that physicians entered were ‘prn’, ‘different time’ or ‘no replacement’, which were judged as being inappropriate. The APM, FM, OBGY and NS departments made the largest contributions to the high overall override rate. The APM physicians had two main kinds of work: pain management and anesthesia during surgical operation. Combinations of QT-prolonging agents and beta-adrenergic blockers or amphetamine derivatives were commonly unaccepted by anesthesiologists, who closely monitor patients using devices throughout surgical operations. The main alerts in FM were NSAIDs–aspirin combinations, while the main overrides in OBGY and NS were NSAID–NSAID combinations to control acute pain, followed by surgical procedures. Intervening in only these combinations could have decreased the frequency of overrides more than 50%. Another particularly interesting finding was that only five drug-class combinations were responsible for almost 90% of all overrides. A study conducted in Switzerland similarly found that four to ten drug pairs accounted for 52–97% of severe overrides [18]. A study conducted in the Netherlands [14] found that 10 and 50 drug-class combinations contributed to >50% and >90% of all DDI alerts, respectively. A common feature of these studies is that the DDI rules were based on national standards that are used in all hospitals. Zwart-van Rijkom et al. [14] explained that their findings were related to the national DDI standard primarily being developed for use in community pharmacies. In our chart review, a large portion of overrides were related to the risks of hyperkalemia and gastrointestinal bleeding (among the five drug-class combinations) which were used to prevent a decrease in serum potassium after cardiac surgery or for postoperative pain control. Hypokalemia is known as a strong independent predictor of mortality in patients with heart failure. Potassium replacement therapy is desirable to avoid hypokalemia proactively by the aggressive replacement of potassium [19]. As for postoperative pain control, use of opioids can be alternatives of NSAID combinations that are associated with a risk of gastrointestinal bleeding. However, postoperative opioid-related respiratory depression is reportedly an important cause of iatrogenic injury in the perioperative period [20]. Therefore, for the inpatient setting those rules should be modified by relevant variables to increase rule specificity which could decrease override rates by at least 50%. Combining the results of the appropriateness chart review and comparing the highest- and lowest-quintile physicians yielded clues for which rules should be modified or conditionally constrained. In addition, some interventions will likely need to target providers with high override rates. For example, for combinations of QT-prolonging agents–beta-adrenergic blockers or amphetamine derivatives which are not used in anesthesia, the most effective interventions might be behavioral ones designed to appeal to provider self-image and social motivations. A study [21] reported that the use of accountable justification and peer comparison as behavioral interventions resulted in lower rates of inappropriate antibiotic prescribing for acute respiratory tract infections. The accountable justification was to prompt physicians to enter a free-text justification for prescribing antibiotics on the EMRs, so that other providers could see it. If he or she did not enter a justification, the statement ‘No justification for prescribing antibiotics was given’ would be entered into the EMR. The present results must be interpreted within the context of the study design. This study has two limits. First, it was conducted in a single center and focused on one type of medication alert of DDI, so our results might not be generalizable to healthcare systems such as community health systems, subacute hospitals or other tertiary hospitals with different DDI rule sets of government or commercial products in other countries. Second, this study has a cross-sectional design, so it is hard to establish a causal relationship. For the external validity, future studies are needed with attempting to identify how such habituation occurs over time and who is vulnerable to it. The strength of this study is that we investigated physicians’ behavior patterns and variation, which is the lack of studies in this field. Conclusions We found a high override rate on DDI alerts and wide variation at the levels of physicians, medical departments and drug classes. These variations are interrelated and provide clues about which requirements are not met by the DDI alert system and which rules may have a low specificity in the clinical context of different medical departments. Two kinds of physician response on alerts contributed to the wide variation: habitual and perceptual overrides. Efforts to improve the acceptance rate of DDIs should be initiated and designed based on a systematic understanding of the behaviors and patterns physicians exhibit in accepting and over-riding decision support. Funding This research was supported by a Korea Research Foundation Grant, which was funded by the Korean Government (MOEHRD; No. NRF-2016R1D1A1A09919502). D.B. was sponsored by the Centers for Education and Research on Therapeutics (CERT grant number 1U19HS021094-01), Agency for Healthcare Research and Quality, Rockville, MD, USA. References 1 McEvoy DS, Sittig DF, Hickman T-T et al.  . Variation in high-priority drug–drug interaction alerts across institutions and electronic health records. J Am Med Inform Assoc  2016; 24: 331– 8. 2 Phansalkar S, Desai AA, Bell D et al.  . High-priority drug–drug interactions for use in electronic health records. J Am Med Inform Assoc  2012; 19: 735– 43. Google Scholar CrossRef Search ADS PubMed  3 Paterno MD, Maviglia SM, Gorman PN et al.  . Tiering drug–drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc  2009; 16: 40– 6. Google Scholar CrossRef Search ADS PubMed  4 Seidling HM, Klein U, Schaier M et al.  . What, if all alerts were specific–estimating the potential impact on drug interaction alert burden. Int J Med Inform  2014; 83: 285– 91. Google Scholar CrossRef Search ADS PubMed  5 Eschmann E, Beeler PE, Schneemann M et al.  . Developing strategies for predicting hyperkalemia in potassium-increasing drug–drug interactions. J Am Med Inform Assoc  2017; 24: 60– 6. Google Scholar CrossRef Search ADS PubMed  6 Duke JD, Li X, Dexter P. Adherence to drug—drug interaction alerts in high-risk patients: a trial of context-enhanced alerting. J Am Med Inform Assoc  2013; 20: 494– 8. Google Scholar CrossRef Search ADS PubMed  7 Schreiber R, Gregoire JA, Shaha JE et al.  . Think time: a novel approach to analysis of clinicians’ behavior after reduction of drug–drug interaction alerts. Int J Med Inform  2017; 97: 59– 67. Google Scholar CrossRef Search ADS PubMed  8 Bryant A, Fletcher G, Payne T. Drug interaction alert override rates in the meaningful use era: no evidence of progress. Appl Clin Inform  2014; 5: 802– 13. Google Scholar CrossRef Search ADS PubMed  9 Slight SP, Seger DL, Nanji KC et al.  . Are we heeding the warning signs? Examining providers’ overrides of computerized drug–drug interaction alerts in primary care. PLoS ONE  2013; 8: e85071. Google Scholar CrossRef Search ADS PubMed  10 Baysari MT, Tariq A, Day RO et al.  . Alert override as a habitual behavior—a new perspective on a persistent problem. J Am Med Inform Assoc  2017; 24: 409– 12. Google Scholar PubMed  11 Tamblyn R, Reidel K, Patel V. Physicians’ response to computerised alerts for psychotropic drugs in older persons: a multilevel analysis of the associated alert, patient and physician characteristics. BMJ Open  2012; 2: e001384. Google Scholar CrossRef Search ADS PubMed  12 Cho I, Slight SP, Nanji KC et al.  . The effect of provider characteristics on the responses to medication-related decision support alerts. Int J Med Inform  2015; 84: 630– 9. Google Scholar CrossRef Search ADS PubMed  13 Nanji KC, Slight SP, Seger DL et al.  . Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc  2014; 21: 487– 91. Google Scholar CrossRef Search ADS PubMed  14 Zwart-van Rijkom JEF, Uijtendaal EV, Ten Berg MJ et al.  . Frequency and nature of drug–drug interactions in a Dutch university hospital. Br J Clin Pharmacol  2009; 68: 187– 93. Google Scholar CrossRef Search ADS PubMed  15 Simpao AF, Ahumada LM, Desai BR et al.  . Optimization of drug–drug interaction alert rules in a pediatric hospital’s electronic health record system using a visual analytics dashboard. J Am Med Inform Assoc  2015; 22: 361– 9. Google Scholar PubMed  16 Cho I, Slight SP, Nanji KC et al.  . Understanding physicians’ behavior toward alerts about nephrotoxic medications in outpatients: a cross-sectional analysis. BMC Nephrol  2014; 15: 200. Google Scholar CrossRef Search ADS PubMed  17 Zhang Y, Baicker K, Newhouse JP. Geographic variation in the quality of prescribing. N Engl J Med  2010; 363: 1985– 8. Google Scholar CrossRef Search ADS PubMed  18 Oertle M. Frequency and nature of drug–drug interactions in a Swiss primary and secondary acute care hospital. Swiss Med Wkly  2012; 142: w13522. Google Scholar PubMed  19 Merchant Q, Rehman Siddiqui NU, Rehmat A et al.  . Comparison of Enteral versus Intravenous Potassium Supplementation in hypokalaemia in postcardiac surgery paediatric cardiac intensive care patients: prospective open label randomised control trial (EIPS). BMJ Open  2014; 4: e005124. Google Scholar CrossRef Search ADS PubMed  20 Lee LA, Caplan RA, Stephens LS et al.  . Postoperative opioid-induced respiratory depression: a closed claims analysis. Anesthesiology  2015; 122: 659– 65. Google Scholar CrossRef Search ADS PubMed  21 Meeker D, Linder JA, Fox CR et al.  . Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial. JAMA  2016; 315: 562– 70. Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press in association with the International Society for Quality in Health Care. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

International Journal for Quality in Health CareOxford University Press

Published: May 8, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off