BackgroundAlthough computerized physician order entry reduces medication errors among inpatients, little is known about the use of this system in primary care.MethodsWe calculated the override rate among 3481 consecutive alerts generated at 5 adult primary care practices that use a common computerized physician order entry system for prescription writing. For detailed review, we selected a random sample of 67 alerts in which physicians did not prescribe an alerted medication and 122 alerts that resulted in a written prescription. We identified factors associated with the physicians' decisions to override a medication alert, and determined whether an adverse drug event (ADE) occurred.ResultsPhysicians overrode 91.2% of drug allergy and 89.4% of high-severity drug interaction alerts. In the multivariable analysis using the medical chart review sample (n = 189), physicians were less likely to prescribe an alerted medication if the prescriber was a house officer (odds ratio [OR], 0.26; 95% confidence interval [CI], 0.08-0.84) and if the patient had many drug allergies (OR, 0.70; 95% CI, 0.53-0.93). They were more likely to override alerts for renewals compared with new prescriptions (OR, 17.74; 95% CI, 5.60-56.18). We found no ADEs in cases where physicians observed the alert and 3 ADEs among patients with alert overrides, a nonsignificant difference (P= .55). Physician reviewers judged that 36.5% of the alerts were inappropriate.ConclusionsFew physicians changed their prescription in response to a drug allergy or interaction alert, and there were few ADEs, suggesting that the threshold for alerting was set too low. Computerized physician order entry systems should suppress alerts for renewals of medication combinations that patients currently tolerate.IN THE Adverse Drug Event Prevention Study, computerized physician order entry (CPOE) prevented up to 84% of medication errors among patients hospitalized at 2 academic medical centers.On the strength of this evidence, CPOE was heralded as a hospital "best practice" in medication safety and a litmus test of safe care.However, little is known about whether CPOE can prevent adverse drug events (ADEs), defined as injuries due to medications, in ambulatory care.In published studies to date, investigatorsreported that rudimentary computerized prescribing at 2 primary care practices resulted in substantially fewer prescription-writing errors than at similar sites without CPOE. Differences were due to legibility and prompting by the automated systems to provide complete prescription information. However, they found no statistically significant difference between ADE rates at computerized and handwritten sites. The lack of a difference was attributed to the small sample size and lack of advanced features such as drug interaction and drug allergy checking at the computerized sites. Many of the preventable ADEs were due to prescription of drugs to which the patient had a known allergy.Can advanced CPOE features alter physician behavior and reduce ADEs in primary care? The answer depends in part on whether physicians honor the computer's drug alert warning. In inpatient electronic order-writing systems, some clinicians discount, ignore, or circumvent alerts and reminders.Clinicians may forfeit the potential benefit of automated decision support if they find the system annoying, unhelpful, or inefficient.To assess the potential benefits of computerized prescribing among ambulatory patients, we examined the behavior of general internists with respect to high-severity drug interaction and drug allergy alerts generated by a CPOE system used at adult primary care practices. We hypothesized that most physicians would honor high-severity drug interaction and drug allergy alerts, and that failure to do so would lead to an increase in ADEs. We sought to examine attributes of patient, prescriber, and medication that affected physicians' decisions to override an alert.METHODSSTUDY SITEThis study included physicians at 5 adult primary care practices affiliated with Beth Israel Deaconess Medical Center (BIDMC), a Boston teaching hospital. Two sites are hospital-based clinics together employing 35 full- and part-time academic internists and 158 medical house officers. Three sites are community-based practices employing a total of 19 full-time internists. One community site serves an affluent suburban community; another is a hospital-owned multispecialty group practice; the third is a community health center. Attending physicians and house officers at the 5 study sites care for a diverse population of more than 50 000 patients.CPOE AND THE ONLINE MEDICAL RECORDPhysician informaticists at BIDMC and the Harvard Center for Clinical Computing, Boston, developed a variety of advanced applications for inpatients and outpatients since the 1970s.The Online Medical Record (OMR) was introduced in 1989 in hospital-based primary care practices, but its use spread quickly throughout BIDMC and its community sites.The OMR provides clinicians with features, including note-writing capability; access to information about laboratory, pathology, and radiology reports and a reminder system for routine screening and preventive health measures. The OMR includes a medication sheet that permits clinicians to submit drug allergy information and to enter and print prescriptions. The system also relies on clinicians to keep medication and allergy information up-to-date. Providers have ready electronic access to the presciption history of their patients. Physicians, nurses, and pharmacists may enter drug allergy and intolerance information into the system; the computer prompts the clinician to record the type of reaction and level of certainty. The prescription-writing feature offers prescribers a list of medications and available strengths, with required fields for drug dose, number of pills or units dispensed, and renewals. Clinicians enter the directions as free text. In 2000, 888 clinicians in 67 practices ordered 296 539 new prescriptions using the OMR.In February 2000, the OMR prescription-writing program was enhanced to include drug interaction and drug allergy checks for all prescription orders. Allergy alerts were generated by linking a central database that includes information about each patient's drug allergies and intolerances that was entered by nurses, physicians, and pharmacists in the inpatient, ambulatory, and home care settings. The allergy program triggers an alert if the prescription matches the brand or generic name of the drug or allergen group defined by the National Drug Data File of First Databank, Inc, San Bruno, Calif. Drug interaction alerts were generated by checking patients' electronic medication list using rules in the National Drug Data File of First Databank. First Databank identifies 3 drug interaction severity levels. Level 1 alerts indicate the potential for serious or life-threatening injury and are based on substantial empirical evidence. Level 2 alerts indicate the potential for less serious injury. Level 3 alerts, like level 1 alerts, indicate the potential for serious or life-threatening injury, but the evidence of the interaction is less compelling. A single prescription could generate a drug allergy alert and multiple drug interaction alerts.When an alert is generated, a note appears on the computer screen that identifies the alerted drug, the drug interaction or allergy, and an indication of the severity of drug interactions (level 1 indicates high; level 2, medium; and level 3, low) (Figure 1and Figure 2). The system also allows the provider to see a detailed, referenced monograph about the alert. To escape from the alert, the prescriber must select deliberately the "override" menu option; the default is to terminate the order.Figure 1.Sample drug alert notification using the Online Medical Record. Hcl indicates hydrochloride; Celexa, citalopram hydrobromide; SSRI, selective serotonin reuptake inhibitor; trazodone, trazodone hydrochloride; and 5-HT, serotonin.Figure 2.Drug allergy and drug interaction alerts in primary care, by physicians' decisions to override. Hcl indicates hydrochloride; SSRI, selective serotonin reuptake inhibitor; trazodone, trazodone hydrochloride; and 5-HT, serotonin.STUDY DESIGN AND SAMPLE SELECTIONThe study was a retrospective series. The outcome was the primary care physician's decision whether to write a prescription for a medication flagged for a level 1, 2, or 3 drug interaction or a drug allergy alert.After obtaining institutional review board approval, we downloaded all 7877 drug interaction and drug allergy alerts generated from October 1 through December 31, 2000. This study period provided a 9-month interval from the implementation of the enhanced CPOE system to allow clinicians (other than some house officers or new hires) to become accustomed to the upgrade. The file included patients' names and hospital identification numbers, name of the medication that generated an alert, date of the alert, type of alert (drug interaction or drug allergy), specific drug-drug interaction(s) associated with a medication, severity of the alert (levels 1-3), and prescriber. The file also included information from the hospital's electronic credentialing system such as physician specialty, sex, and year of medical school graduation.We reviewed initially all 4751 alerts generated by Department of Medicine internists. We found that physicians frequently tried several times to write a prescription for an alerted medication. They viewed a monograph about the alert, exited from the ordering process using an escape keystroke, then attempted to reorder the medication. This behavior occurred in 1270 of the 4751 alerts (26.7%). To avoid double counting, we selected the last in a sequence of escapes or overrides for a particular physician, drug, patient, and date. This yielded 3481 alerts.In the initial analysis of all 3481 alerts generated by Department of Medicine internists, we calculated how frequently physicians overrode drug allergy and drug interaction alerts. In the subsequent analysis, we selected a subset of 189 alerts for detailed medical chart review. The medical chart review sample was limited to alerts that triggered drug allergy and level 1 drug interaction alerts, which together represented the greatest potential for harm. We randomly selected 122 alerts (31 drug allergy and 91 drug interaction) from 834 alerts in which the physician chose to override the alert and to complete a written prescription (14.6% of the sample). We randomly selected 67 alerts (15 drug allergy and 52 drug interaction) from 92 alerts where the physician did not prescribe an alerted medication (72.8% of the sample).MEDICAL CHART REVIEWAn internist (M.T.) reviewed the electronic medical record for the 3-month period from the date an alert was triggered. The reviewer completed a medical chart review instrument and printed the patient's medication sheet and the office note from the date of the index visit. She abstracted demographic information, including the patient's age, sex, and type of insurance. She indicated whether a prescription for the alerted drug was in fact written, the type of encounter (telephone or office visit), and whether the prescribing physician commented in the medical record about the alert. She assessed whether the prescription was for a new medication, a new dose of an existing prescription, or a renewal. She identified the presence of possible ADEs. She documented the name and number of drug allergies, the number of prescription medications, and the number of items on the patient's OMR problem list. A second internist (S.N.W.) reviewed the completed medical chart review instruments and related materials for completeness and accuracy.PHYSICIAN REVIEWTwo board-certified internists (D.Z.S. and M.D.A.) independently reviewed each case to determine whether the alert was valid on the basis of scientific data, published drug information, and clinical utility (ie, a good alert), and whether the prescribing physician's decision to override the alert was appropriate given the clinical scenario (ie, a good decision). For events that resulted in an injury to a patient, the reviewers determined whether an ADE was present and whether the ADE resulted from failure to observe a computer alert. Physician reviewers scored each item on a 4-point scale (definite, probable, possible, and unlikely). Differences were resolved by consensus. Interrater agreement regarding the validity of the alert was excellent (κ = 0.86) and regarding the appropriateness of the physicians' decision to override an alert was good (κ = 0.66).DATA ANALYSISTo evaluate the hypothesis that most physicians accept drug allergy and high-severity drug interaction alerts, we calculated the number and percentage of each type of alert physicians overrode. Next, using the medical chart review sample, we tabulated the drugs, allergies, and types of interaction that generated level 1 interaction or allergy-related computer alerts. We examined the relationship between the decision to prescribe and factors hypothesized to affect physicians' decisions. We included demographic factors, such as the age and sex of patient and physician (using years since graduation from medical school as a proxy for age in the latter group), and insurance type (as a proxy for patients' socioeconomic status). We included the number of medication allergies per patient, number of prescription medications, and number of medical problems present on the patient's problem list. We identified the level and type of physician practice (house officer, community-based physician, or hospital staff physician). We accounted for the type of alert (drug allergy or drug interaction), type of encounter (office visit or telephone contact), and new or renewed prescription.We hypothesized that house officers would be more receptive to electronic alerts than more experienced physicians. We hypothesized that physicians would be less likely to override alerts in patients with multiple allergies, who were receiving many medications, or who had many comorbid medical illnesses. In these cases, physicians may have fewer available choices. We hypothesized that physicians would override more alerts for new prescriptions than for a changed dose or renewed prescription of a medication that the patient previously tolerated.We used the χ2statistic for comparisons involving categorical variables and the Fisher exact test to analyze ADE rates. We used a multivariable logistic regression model with forward selection (P<.20) to identify factors associated with physicians' decision to write a prescription that generated an alert. Analyses used Stata 7.0 software (Stata Corp, College Station, Tex).RESULTSSUMMARY OF ALERTSDuring the 3-month study period, primary care physicians at the 5 study sites attempted to write 24 034 prescriptions using the OMR. These prescriptions generated 352 drug allergy and 3129 drug interaction alerts (after correcting for redundant keystrokes) for a total of 3481 alerts (Table 1). Primary care physicians overrode 91.2% of 352 drug allergy alerts, 89.4% of 574 level 1 drug interaction alerts, 96.3% of 2432 level 2 alerts, and 85.4% of 123 level 3 alerts. To understand factors that influenced prescribing behavior, we reviewed a sample of 189 alerts for detailed analysis.Table 1. Drug Allergy and Drug Interaction Alerts in Primary Care, by Physicians' Decisions to OverrideNo. (%) of AlertsOverride (n = 3280)No Override (n = 201)Total (N = 3481)Drug allergy321 (91.2)31 (8.8)352 (10.1)Drug interaction level2959 (94.6)170 (5.4)3129 (89.9)1 (High severity, strong evidence)513 (89.4)61 (10.6)574 (16.5)2 (Moderate severity, strong evidence)2341 (96.3)91 (3.7)2432 (69.9)3 (High severity, moderate evidence)105 (85.4)18 (14.6)123 (3.5)Total Alerts3280 (94.2)201 (5.8)3481 (100.0)RESPONSIBLE DRUGSIn the medical chart review sample, 96 drugs generated 143 level 1 drug interaction alerts and 35 drugs generated 46 drug allergy alerts. The following 6 drugs accounted for approximately one third of alerts: cyclobenzaprine hydrochloride (16%), azithromycin (6%), atenolol (3%), a combination of trimethoprim and sulfamethoxazole (3%), clarithromycin (3%), and a combination of triamterene and hydrochlorthiazide (3%).DRUG ALLERGY ALERTSAntibiotics, cardiovascular medications, and analgesics accounted for 42 of 46 drug allergy alerts. Physicians overrode the analgesic allergy alert in 13 of 14 cases. In each case, the alert was triggered when a physician wrote a prescription for a drug that was in the same class as the drug to which the patient had an allergy. For example, hydrocodone bitartrate and oxycodone hydrochloride prescriptions triggered allergy alerts to codeine phosphate. Similarly, ibuprofen triggered an alert to naproxen.In contrast, physicians overrode only 7 of 17 antibiotic allergy alerts. Prescriptions for amoxicillin and a combination of amoxicillin and clavulanate potassium triggered 6 penicillin allergy alerts; all but one was honored. In contrast, physicians overrode all 3 erythromycin allergy alerts triggered by prescriptions for azithromycin (all involved erythromycin intolerance rather than a true allergy). Overall, physicians overrode 31 of 46 drug allergy alerts included in the medical chart review.DRUG INTERACTION ALERTSFifty-three different drug interactions triggered 143 drug interaction alerts. Four drug interactions accounted for 65 of 143 level 1 alerts. The sympathomimetic–tricyclic antidepressant interaction was triggered most often (43 of 143), followed by anticoagulant-macrolide and selective serotonin reuptake inhibitor–tricyclic interactions (8 of 143 for each), then triamterene- or amiloride hydrochloride–nonsteroidal anti-inflammatory drug interactions (6 of 143). Cyclobenzaprine accounted for more than half (24 of 43) of the sympathomimetic–tricyclic antidepressant interaction alerts and 6 of 8 selective serotonin reuptake inhibitor–tricyclic interaction alerts; azithromycin accounted for 6 of 8 anticoagulant-macrolide interaction alerts. Overall, physicians overrode 91 of 143 drug interaction alerts included in the medical chart review.PATIENT AND PHYSICIAN ATTRIBUTES ASSOCIATED WITH PHYSICIANS' DECISIONSWe compared cases in which the primary care physician did and did not write a prescription for an alerted medication by factors hypothesized to affect physicians' decisions (Table 2). In the univariable analysis, the only factor associated with the decision to write a prescription for an alerted medication was prescription type. New prescriptions were less often written than renewals (50.0% vs 89.6% among alerted medications; P<.001).Table 2. Attributes Associated With Prescription of an Alerted Medication*No. (%) of AlertsTotal No. (N = 189)P ValueWritten (n = 122)Not Written (n = 67)Patient attributesSexMale39 (32.0)17 (25.4)56.34Female83 (68.0)50 (74.6)133Age, y<3510 (8.2)9 (13.4)19.3335-6576 (62.3)35 (52.2)111>6536 (29.5)23 (34.3)59Insurance typeManaged care64 (52.5)26 (38.8)90.31Medicare37 (30.3)27 (40.3)64Medicaid, free care, and self-pay13 (10.7)11 (16.4)24Commercial6 (4.9)2 (3.0)8Unknown2 (1.6)1 (1.5)3No. of medical problems<534 (27.9)16 (23.9)50.735-836 (29.5)21 (31.3)57/PARA>9-1230 (24.6)14 (20.9)44>1222 (18.0)16 (23.9)38No. of drug allergies048 (39.3)25 (37.3)73.721-251 (41.8)26 (38.8)77>223 (18.9)16 (23.9)39No. of prescription medications<518 (14.8)5 (7.5)23.385-834 (27.9)19 (28.4)539-1228 (23.0)21 (31.3)49>1242 (34.4)22 (32.8)64Physician attributesSexMale65 (53.3)38 (56.7)103.65Female57 (46.7)29 (43.3)86Sex concordance with patientSame70 (57.4)36 (53.7)106.63Different52 (42.6)31 (46.3)83Years since graduation≤1046 (37.7)30 (44.8)76.2311-2044 (36.1)16 (23.9)60>2032 (26.2)21 (31.3)53Level and type of experienceStaff physician39 (32.0)13 (19.4)52.29Community-based physician58 (47.5)40 (59.7)98House officer/fellow25 (20.5)14 (20.9)39Type of encounterTelephone22 (18.0)10 (14.9)32.59Office visit100 (82.0)57 (85.1)157Type of prescription and alertPrescription typeNew prescription61 (50.0)60 (89.6)121<.001Renewal with change15 (12.3)015Renewal, no change46 (37.7)7 (10.4)53Alert typeDrug-drug interaction91 (74.6)52 (77.6)143.64Drug allergy31 (25.4)15 (22.4)46No. of alerts per prescription1105 (86.1)55 (82.1)160.52>117 (13.9)12 (17.9)29Alerted drugCyclobenzaprine16 (13.1)14 (20.9)30.16Other106 (86.9)53 (79.1)159Interaction typeSympathomimetic–tricyclic antidepressant29 (23.8)14 (20.9)43.65Other93 (76.2)53 (79.1)146*Percentages have been rounded and may not total 100.In the multivariable logistic regression model (Table 3), physicians were less likely to prescribe (override) an alerted medication if the patient had multiple medication allergies (odds ratio [OR], 0.70; 95% confidence interval [CI], 0.53-0.93), and if the prescriber was a house officer (compared with a staff physician) (OR, 0.26; 95% CI, 0.08-0.84). Physicians were substantially more likely to override an alert for a renewal of a current prescription than for a new prescription (OR, 17.74; 95% CI, 5.60-56.18).Table 3. Multivariable Analysis of Factors Associated With Physician Prescription of an Alerted Medication*FactorAdjusted OR (95% CI)Patient attributesMedicare insurance (vs managed care)0.48 (0.21-1.10)No. of medication allergies0.70 (0.53-0.93)Physician attributesMale0.48 (0.22-1.07)Community-based physician0.41 (0.17-1.03)House officer (vs staff physician)0.26 (0.08-0.84)Encounter, prescription, and alert attributesRenewal (vs new prescription)17.74 (5.60-56.18)Abbreviations: CI, confidence interval; OR, odds ratio.*Calculated using multivariable logistic regression model with forward selection (P<.20).PHYSICIAN EXPLANATIONSThirty-two of 185 physicians in the study commented in the electronic medical record about their decision to observe or ignore a medication alert. In 15 of the 22 cases in which a physician had prescribed an alerted medication and commented about the decision, the physician wrote that the patient was not currently taking the medication listed as a potential cause of a level 1 interaction because it had been discontinued or the course had been completed. In 3 cases, the physician wrote that use of the prescribed medication was justified for a limited duration. In 2 more cases, the alert was prompted by a prescription of a medication with a listed allergy, but the drug was reported to be tolerated by the patient in the past. In the remaining 2 cases of allergy alerts, the physician indicated that the medication sheet or allergy information was incorrect.REVIEWER ASSESSMENT OF ALERTS AND PHYSICIANS' DECISIONSTo assess independently the prescribers' decisions, 2 investigators reviewed each alert. Reviewers judged that 69 (36.5%) of 189 alerts were invalid (including 58 [40.6%] of 143 drug interaction and 11 [23.9%] of 46 drug allergy alerts). They judged that 37 (86.0%) of 43 sympathomimetic–tricyclic antidepressant interaction alerts, none of 8 anticoagulant-macrolide interaction alerts, none of 8 selective serotonin reuptake inhibitor–tricyclic interaction alerts, and none of 6 potassium-sparing diuretic–nonsteroidal anti-inflammatory drug interaction alerts were unjustified on the basis of scientific evidence.Reviews also examined the prescribers' decisions to override an alert. Reviewers agreed with the prescribers' decisions in 185 (97.9%) of 189 cases, including 65 (95.6%) of 68 cases where the physician chose to override a valid alert. Many of these alerts provided worthwhile warnings. For example, most anticoagulant and antibiotic alerts were judged to be valid but appropriate to override (especially if a dose adjustment was recommended or follow-up laboratory tests were ordered). For each of the 119 cases in which a physician wrote a prescription that the reviewers judged appropriate, the reviewers indicated at least 1 reason to justify the decision (Table 4). Reviewers indicated most often that the patient was no longer taking the medication, the interaction was not clinically significant, the patient tolerated the drug(s), and the benefits of treatment outweighed the disadvantages.Table 4. Appropriate Prescriptions for Alerted Medications: Reviewer Justifications (n = 119)ReasonNo. (%)Alerted interaction not clinically significant35 (21.6)Patient currently tolerates the medication or combination35 (21.6)Benefit of drug outweighs disadvantages34 (21.0)Patient has tolerated medication or combination in the past20 (12.3)Medication list out of date13 (8.0)Limited course of treatment10 (6.2)No good alternatives available to alerted medication6 (3.7)Allergy information inaccurate in patient's record3 (1.9)Other6 (3.7)Total*162 (100.0)*Total exceeds 119 because reviewers were permitted to identify more than 1 justification per alerted medication.ADVERSE DRUG EVENTSWe identified 3 ADEs in the study, all affecting patients whose physicians had prescribed a drug that generated a level 1 drug interaction alert. One involved supratherapeutic anticoagulation (international normalized ratio, 8) in a patient receiving warfarin sodium for an aortic valve replacement and atrial fibrillation; the patient was prescribed an antibiotic (clarithromycin) for a lower respiratory tract infection, but had no bleeding complications. In another case, epistaxis developed that required an emergency department visit. The patient took warfarin for atrial fibrillation and was prescribed aspirin (325 mg/d) 3 weeks earlier for coronary artery disease. The international normalized ratio was in the therapeutic range, and the hematocrit level was stable. Reviewers judged that both ADEs were related to the alerted medications. In a third case, clonidine hydrochloride patch therapy was discontinued 1 month after it was started because the patient complained of itching at the site of the patch. The clonidine prescription had triggered an interaction alert with β-blockers; the ADE was judged unrelated to the interaction.An ADE affected 3 (2.5%) of 122 patients whose physician had written a prescription for an alerted medication and none of the 67 patients whose physicians had observed the alert, a difference that did not reach statistical significance (P= .55). Reviewers judged only the first event (the warfarin-clarithromycin interaction) as an event that was potentially preventable had the prescribing physician adjusted the anticoagulant dosage in advance.COMMENTIn this study of medication safety, we set out to examine whether primary care internists honor drug allergy and drug interaction alerts generated by a CPOE system, and to understand the factors that influenced physicians' behavior. Physicians overrode alerts in most cases.This result is consistent with a study of alerts in the inpatient setting, in which physicians observed fewer than half of potentially life-threatening drug interaction alerts.However, clinicians' responses to medication alerts were less frequent than the responses reported in other studies of alerts and reminders in the inpatient setting. In one of the earliest studies, McDonaldand other investigatorsshowed that physicians responded to 51% of computer-generated reminders about condition management and drug toxicities; 22% of physicians in the control group responded to similar conditions without the benefit of a reminder. The effect is also smaller than the compliance rates of 15% to 30% for preventive health reminders in ambulatory care.To understand the factors that influenced physicians' decisions to write a prescription for an alerted medication, we sampled from alerts that physician had and had not overridden. We found that certain drugs (eg, cyclobenzaprine) and drug interactions (eg, tricyclic antidepressants and sympathomimetics) accounted for a disproportionate share of alerts. Physicians were more likely to override some alerts than others; this behavior may signal interactions that clinicians view with suspicion. In fact, physician reviewers judged that one third of alerts were inappropriate.When we examined factors hypothesized to affect physicians' decision to prescribe an alerted medication, we found that physicians less often overrode alerts among patients with multiple drug allergies, suggesting that there were few therapeutic alternatives available. In addition, house officers were less likely than hospital-based faculty internists to override alerts. As novice physicians, house officers may be receptive to new information and to the introduction of technology into their practice. Rejection of alerts by community physicians and faculty physicians may reflect the skepticism of physicians with greater experience about some features of the CPOE system, such as out-of-date information, identification of interactions that were not clinically significant, failure to note patient tolerance of medication combinations, and the inability to balance the risks and benefits of therapy. It may also reflect a deeper-seated resistance among experienced practitioners to the perceived intrusion of information technology into the practice of clinical medicine.Clinicians' acceptance of alerts and reminders reflects their relevance and validity. For example, Raschke and colleaguesfound that only 53% of 1116 medication alerts generated by a CPOE system at a 650-bed Arizona community teaching hospital accurately identified hazardous situations. A high volume of inappropriate or unhelpful alerts may discourage busy clinicians' willingness to examine, consider, and act on them.The Adverse Drug Event Prevention study by Bates et alshowed that a sophisticated CPOE system can prevent medication errors among inpatients. Will CPOE prevent ADEs in primary care? We do not yet know which components of decision support in CPOE systems are necessary to produce improvements in primary care. Checking for drug allergy and interactions may be necessary but not sufficient to improve medication safety. Our results suggest that CPOE designers need to identify and eliminate inappropriate alerts that physicians find incredible, and change the threshold for generating alerts on renewals of medications that patients currently tolerate in combination. In addition, they should design advanced algorithms that ensure up-to-date and clinically relevant information is available to clinicians about laboratory data, dosing schedules, adverse drug reactions, and contraindications.This study was subject to several limitations, including its small sample size and retrospective design. It was completed at 5 primary care practices affiliated with a single academic medical center using a single, shared CPOE system. The results may reflect in part features of the patients, clinicians, or information system peculiar to the study site and, hence, may not be generalizable to other practices and other systems. Our data suggest that ADEs occurred more often in cases where the physician failed to observe an alert, but the study was not adequately powered to address this question. We may have underestimated the number and type of ADEs because the medical chart review was limited to 3 months of follow-up. Since patients may fail to report or physicians to record ADE information in the medical record, more aggressive approaches to incident detection might also increase the yield of ADEs. In addition, the relatively short time frame of our study did not allow us to determine whether the override rate increases in a secular fashion. Physicians may increasingly override the system over time, if they perceive the alerts are usually inappropriate.Nevertheless, the study offers information useful to the design of medication safety programs in primary care. Although a minority of primary care internists changed their prescription in response to an alert, most of these decisions were appropriate. Clinicians' skepticism about some CPOE medication alerts appears justified and indicates the importance of developing alert systems that are clinically useful and produce fewer unnecessary alerts. Specifically, systems should suppress alerts for renewals of previously tolerated medication combinations. Systems should prompt providers for information about the level of certainty associated with drug allergy alerts and to distinguish intolerance of well-recognized side effects from IgE-mediated responses. Systems should improve the signal-to-noise ratio by suppressing alerts with little evidentiary basis or clinical relevance and, in turn, provide access to references that can inform physician decisions.In addition, systems should prompt providers to review medication lists for out-of-date information. They should also prompt clinicians to explain the basis for the decision to override an alert, so that this information can be used to inform the development of more effective decision support. Failure to document reasons for overrides may conceal sound clinical reasoning and create a liability risk for clinicians. In short, the development of more sophisticated decision support is needed to realize the promise of CPOE in primary care.DWBatesLLLeapeDJCullenEffect of computerized physician order entry and a team intervention on prevention of serious medication errors.JAMA.1998;280:1311-1316.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9794308&dopt=AbstractMassachusetts Coalition for the Prevention of Medical ErrorsMHA best practice recommendations to reduce medication errors.Available at: http://www.macoalition.org/documents/Best_Practice_Medication_Errors.pdf. Accessed August 7, 2003.Leapfrog GroupFact sheet.Washington, DC: Academy Health, June 2003. Available at: http://www.leapfroggroup.org/FactSheets/LF_FactSheet.pdf. Accessed August 7, 2003.TKGandhiSNWeingartDSSegerMedication errors and adverse events in the ambulatory setting [abstract].J Gen Intern Med.2001;16(suppl):133.TKGandhiSNWeingartJBorusAdverse drug events in ambulatory care.N Engl J Med.2003;348:1556-1564.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=12700376&dopt=AbstractJFPetersonGKupermanCShekDWBatesPhysician responses to life-threatening drug-drug interaction alerts [abstract].J Gen Intern Med.2001;16(suppl):212.WMTierneyMEMillerJMOverhageCJMcDonaldPhysician inpatient order writing on microcomputer workstations.JAMA.1993;269:379-383.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=8418345&dopt=AbstractCSafranElectronic medical records.JAMA.2001;285:1766.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11277836&dopt=AbstractCJMcDonaldProtocol-based computer reminders: the quality of care and the non-perfectability of man.N Engl J Med.1976;295:1351-1355.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=988482&dopt=AbstractGJKupermanJMTeichMJTanasijevicImproving response to critical laboratory results with automation: results of a randomized controlled trial.J Am Med Inform Assoc.1999;6:512-522.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=10579608&dopt=AbstractGJKupermanJMTeichDWBatesDetecting alerts, notifying the physician, and offering action items: a comprehensive alerting system.Proc AMIA Annu Fall Symp.1996:704-708.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=8947756&dopt=AbstractCJMcDonaldSLHuiDMSmithReminders to physicians from an introspective computer medical record.Ann Intern Med.1984;100:130-138.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=6691639&dopt=AbstractWMTierneySLHuiCJMcDonaldDelayed feedback of physician performance versus immediate reminders to perform preventive care: effects on physician compliance.Med Care.1986;24:659-666.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=3736141&dopt=AbstractCJMcDonaldRMurrayDJerisBBhargavaJSeegerLBlevinsA computer-based record and clinical monitoring system for ambulatory care.Am J Public Health.1977;67:240-245.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=842761&dopt=AbstractCJMcDonaldSLHuiWMTierneyEffects of computer reminders for influenza vaccination on morbidity during influenza epidemics.MD Comput.1992;9:304-312.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=1522792&dopt=AbstractRARaschkeBBollihareTAWunderlichA computer alert system to prevent injury from adverse drug events.JAMA.1998;280:1317-1320.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9794309&dopt=AbstractSAAbookireJMTeichHSandigeImproving allergy alerting in a computerized physician order entry system.Proc AMIA Symp.2000;:2-6.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11080034&dopt=AbstractJMOverhageSPerkinsWMTierneyCJMcDonaldControlled trial of direct physician order entry.J Am Med Inform Assoc.2001;8:361-371.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11418543&dopt=AbstractDMRindCSafranRSPhillipsEffect of computer-based alerts on the treatment and outcomes of hospitalized patients.Arch Intern Med.1994;154:1511-1517.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=8018007&dopt=AbstractGJKupermanDWBatesJMTeichJRSchneiderDCheimanA new knowledge structure for drug-drug interactions.Proc Annu Symp Comput Appl Med Care.1994;:836-840.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=7950042&dopt=AbstractGDSchiffHCAggarwalSKumarRAMcNuttPrescribing potassium despite hyperkalemia.Am J Med.2000;109:494-497.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11042241&dopt=AbstractCorresponding author: Saul N. Weingart, MD, PhD, at the Division of General Medicine and Primary Care, Beth Israel Deaconess Medical Center, Rose 112, 330 Brookline Ave, Boston, MA 02215 (e-mail: email@example.com).Accepted for publication January 23, 2003.This study was funded by a grant from the Stoneman Center for Quality Improvement in General Medicine and Primary Care, Beth Israel Deaconess Medical Center, Boston, Mass, and in part by a K08 Mentored Clinical Scientist Career Development Award (1 K08 HS11644-01) from the US Agency for Healthcare Research and Quality, Rockville, Md (Dr Weingart).This study was presented as an abstract at the 25th Annual Meeting of the Society of General Internal Medicine; May 3, 2002; Atlanta, Ga.We thank David W. Bates, MD, MSc, for his comments on an earlier draft of this article.
JAMA Internal Medicine – American Medical Association
Published: Nov 24, 2003
It’s your single place to instantly
discover and read the research
that matters to you.
Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.
All for just $49/month
Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly
Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.
Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.
Read from thousands of the leading scholarly journals from SpringerNature, Wiley-Blackwell, Oxford University Press and more.
All the latest content is available, no embargo periods.
“Hi guys, I cannot tell you how much I love this resource. Incredible. I really believe you've hit the nail on the head with this site in regards to solving the research-purchase issue.”Daniel C.
“Whoa! It’s like Spotify but for academic articles.”@Phil_Robichaud
“I must say, @deepdyve is a fabulous solution to the independent researcher's problem of #access to #information.”@deepthiw
“My last article couldn't be possible without the platform @deepdyve that makes journal papers cheaper.”@JoseServera