Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

A clinical decision support system algorithm for intravenous to oral antibiotic switch therapy: validity, clinical relevance and usefulness in a three-step evaluation study

A clinical decision support system algorithm for intravenous to oral antibiotic switch therapy:... Abstract Objectives To evaluate a clinical decision support system (CDSS) based on consensus-based intravenous to oral switch criteria, which identifies intravenous to oral switch candidates. Methods A three-step evaluation study of a stand-alone CDSS with electronic health record interoperability was performed at the Erasmus University Medical Centre in the Netherlands. During the first step, we performed a technical validation. During the second step, we determined the sensitivity, specificity, negative predictive value and positive predictive value in a retrospective cohort of all hospitalized adult patients starting at least one therapeutic antibacterial drug between 1 and 16 May 2013. ICU, paediatric and psychiatric wards were excluded. During the last step the clinical relevance and usefulness was prospectively assessed by reports to infectious disease specialists. An alert was considered clinically relevant if antibiotics could be discontinued or switched to oral therapy at the time of the alert. Results During the first step, one technical error was found. The second step yielded a positive predictive value of 76.6% and a negative predictive value of 99.1%. The third step showed that alerts were clinically relevant in 53.5% of patients. For 43.4% it had already been decided to discontinue or switch the intravenous antibiotics by the treating physician. In 10.1%, the alert resulted in advice to change antibiotic policy and was considered useful. Conclusions This prospective cohort study shows that the alerts were clinically relevant in >50% (n = 449) and useful in 10% (n = 85). The CDSS needs to be evaluated in hospitals with varying activity of infectious disease consultancy services as this probably influences usefulness. Introduction One of the most cost-effective and safe objectives of a hospital antimicrobial stewardship programme is the timely switch from intravenous (iv) to oral antibiotic therapy.1 A timely iv to oral switch (IVOS) has many advantages, such as reduced incidence of catheter-related infections and a decreased hospital length of stay.1–3 However, up to two-thirds of patients eligible for an IVOS remain on iv antibiotics longer than necessary.4–6 A possible explanation is that reassessment of iv-started antibiotics is often not done, for instance owing to time constraints and change of staff.7 In addition, different IVOS criteria are being used to determine whether a patient can be switched from iv to oral antibiotic therapy.8 To address the latter issue, an international Delphi procedure was recently performed to reach consensus among international experts on a set of IVOS criteria that have to be met in adult hospitalized patients for a safe IVOS after 48–72 h of iv therapy.9 This resulted in the development of six IVOS switch criteria, operationalized by 16 measurable conditions; e.g. the IVOS switch criterion ‘signs and symptoms related to the infection have to be resolved or improved’ is operationalized by the following two measurable conditions: temperature should be <38.3°C without antipyretics and >36°C (Table 1). Table 1. Clinical rule iv oral antibiotic switch based on operationalized and consensus-based switch criteria9 iv oral antibiotic switch criteria IVOS algorithm A. Vital signs should be good or improving when bad  systolic blood pressure should be stable without inotropics or fluid resuscitation systolic blood pressure >90 mmHg (alert suppressed if inotropics or fluid resuscitation is used) B. Signs and symptoms related to the infection have to be resolved or improved  temperature should be <38.3°C without antipyretics temperature <38.1°C for a duration of 24 h (notification when antipyretics are used)  temperature should be >36°C temperature >36°C for a duration of 24 h C. Gastrointestinal tract has to be intact and functioning patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  absence of the following conditions   malabsorption syndrome   short bowel syndrome   severe gastroparesis   ileus   continuous nasogastric suction D. Oral route should not be compromised patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  no vomiting  patient should be cooperative E. Absence of contra-indicated infections not possible to translate into a computer interpretable formata  adequate antimicrobial concentrations are not achievable at the site of infection by oral administration  absence of the following infections   (severe) sepsis   fasciitis necroticans   CNS infection   Staphylococcus aureus bacteraemia alert suppressed in case of S. aureus bacteraemia   endovascular infection (e.g. endocarditis) F. An oral variant of the antibiotic with good bioavailability has to exist, including other antibiotics with appropriate microbiological profile not possible to translate into a computer interpretable formata iv oral antibiotic switch criteria IVOS algorithm A. Vital signs should be good or improving when bad  systolic blood pressure should be stable without inotropics or fluid resuscitation systolic blood pressure >90 mmHg (alert suppressed if inotropics or fluid resuscitation is used) B. Signs and symptoms related to the infection have to be resolved or improved  temperature should be <38.3°C without antipyretics temperature <38.1°C for a duration of 24 h (notification when antipyretics are used)  temperature should be >36°C temperature >36°C for a duration of 24 h C. Gastrointestinal tract has to be intact and functioning patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  absence of the following conditions   malabsorption syndrome   short bowel syndrome   severe gastroparesis   ileus   continuous nasogastric suction D. Oral route should not be compromised patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  no vomiting  patient should be cooperative E. Absence of contra-indicated infections not possible to translate into a computer interpretable formata  adequate antimicrobial concentrations are not achievable at the site of infection by oral administration  absence of the following infections   (severe) sepsis   fasciitis necroticans   CNS infection   Staphylococcus aureus bacteraemia alert suppressed in case of S. aureus bacteraemia   endovascular infection (e.g. endocarditis) F. An oral variant of the antibiotic with good bioavailability has to exist, including other antibiotics with appropriate microbiological profile not possible to translate into a computer interpretable formata a Except for S. aureus bacteraemia, which was assessed by the algorithm. Assessment of criteria that could not be translated into a computer interpretable format was left to the discretion of the ID specialist. Table 1. Clinical rule iv oral antibiotic switch based on operationalized and consensus-based switch criteria9 iv oral antibiotic switch criteria IVOS algorithm A. Vital signs should be good or improving when bad  systolic blood pressure should be stable without inotropics or fluid resuscitation systolic blood pressure >90 mmHg (alert suppressed if inotropics or fluid resuscitation is used) B. Signs and symptoms related to the infection have to be resolved or improved  temperature should be <38.3°C without antipyretics temperature <38.1°C for a duration of 24 h (notification when antipyretics are used)  temperature should be >36°C temperature >36°C for a duration of 24 h C. Gastrointestinal tract has to be intact and functioning patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  absence of the following conditions   malabsorption syndrome   short bowel syndrome   severe gastroparesis   ileus   continuous nasogastric suction D. Oral route should not be compromised patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  no vomiting  patient should be cooperative E. Absence of contra-indicated infections not possible to translate into a computer interpretable formata  adequate antimicrobial concentrations are not achievable at the site of infection by oral administration  absence of the following infections   (severe) sepsis   fasciitis necroticans   CNS infection   Staphylococcus aureus bacteraemia alert suppressed in case of S. aureus bacteraemia   endovascular infection (e.g. endocarditis) F. An oral variant of the antibiotic with good bioavailability has to exist, including other antibiotics with appropriate microbiological profile not possible to translate into a computer interpretable formata iv oral antibiotic switch criteria IVOS algorithm A. Vital signs should be good or improving when bad  systolic blood pressure should be stable without inotropics or fluid resuscitation systolic blood pressure >90 mmHg (alert suppressed if inotropics or fluid resuscitation is used) B. Signs and symptoms related to the infection have to be resolved or improved  temperature should be <38.3°C without antipyretics temperature <38.1°C for a duration of 24 h (notification when antipyretics are used)  temperature should be >36°C temperature >36°C for a duration of 24 h C. Gastrointestinal tract has to be intact and functioning patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  absence of the following conditions   malabsorption syndrome   short bowel syndrome   severe gastroparesis   ileus   continuous nasogastric suction D. Oral route should not be compromised patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  no vomiting  patient should be cooperative E. Absence of contra-indicated infections not possible to translate into a computer interpretable formata  adequate antimicrobial concentrations are not achievable at the site of infection by oral administration  absence of the following infections   (severe) sepsis   fasciitis necroticans   CNS infection   Staphylococcus aureus bacteraemia alert suppressed in case of S. aureus bacteraemia   endovascular infection (e.g. endocarditis) F. An oral variant of the antibiotic with good bioavailability has to exist, including other antibiotics with appropriate microbiological profile not possible to translate into a computer interpretable formata a Except for S. aureus bacteraemia, which was assessed by the algorithm. Assessment of criteria that could not be translated into a computer interpretable format was left to the discretion of the ID specialist. These operationalized consensus criteria are a first step towards standardized IVOS criteria. However, to improve appropriate antibiotic use in an effective and sustainable manner, more is needed than guidance and instructions.10 A promising stewardship intervention to facilitate a timely IVOS is a clinical decision support system (CDSS) that automatically generates reminding alerts.11,12 Earlier studies on CDSSs have shown that an important pitfall, which can impede the success of a CDSS, is alert fatigue. To prevent alert fatigue a well-designed CDSS with high specificity and sensitivity is crucial. We developed such a CDSS algorithm that is based on the operationalized consensus criteria for IVOS.9 In this study, we validated this CDSS algorithm and assessed its clinical relevance and usefulness in daily clinical practice. Methods Setting This study was conducted at the Erasmus University Medical Centre (Erasmus MC) in Rotterdam in the Netherlands, a 1237 bed tertiary care centre. The Erasmus MC uses an electronic health record (EHR) with integrated computerized prescriber order entry. This system integrates patient data, medication prescriptions, laboratory data, surgical reports and radiology reports. The Department of Medical Microbiology and Infectious Diseases of this hospital provides an active infectious disease (ID) consultancy service, in which ID consultants actively give the attending physicians antibiotic advice. They are also being consulted for antibiotic advice in nearly half of the patients using antibiotic therapy.13 Study population The study population for the technical and the retrospective clinical validation was extracted from the computerized prescriber order entry and consisted of all hospitalized adult patients starting at least one antibacterial drug (ATC code J01) between 1 and 16 May 2013. Adult ICU patients were included and patients on the cardiothoracic ICU and the paediatric and psychiatric wards were excluded. Patients using prophylactic antibiotics were also excluded. Co-trimoxazole at a maximum daily dose of 480 mg and cefazolin given once were always considered prophylaxis. Antibiotics given according to a local prophylaxis protocol were also defined as prophylaxis. This was assessed manually. The same criteria were used for the prospective study. IVOS algorithm A multidisciplinary team, consisting of an ID specialist, clinical microbiologists, hospital pharmacist experienced in decision support, information technology team and a researcher, developed the CDSS algorithm. The consensus-based and operationalized criteria9 were translated into a computer-interpretable format. All adult hospitalized patients with at least one iv antibiotic prescription for a duration of at least 84 h were selected by the algorithm. This relatively long time period was chosen because we wanted to evaluate the contribution of the CDSS on top of usual (switch) care. An alert indicating that the patient could be safely switched was generated when the conditions described in Table 1 were met. A report with all patients with an alert was automatically generated on a daily basis and directed to the ID specialist of the Antibiotic Stewardship Team. Criterion E ‘Absence of contra-indicated infections’ could not be translated into the CDSS algorithm. The same applied to criterion F ‘An oral variant of the antibiotic with good bioavailability has to exist, including other antibiotics with appropriate microbiological profile’. These criteria were left to the discretion of the ID specialist. An overview of the included antipyretics (used to assess criterion B), inotropics and fluid resuscitation (used to assess criterion A) can be found in Table S1 (available as Supplementary data at JAC Online). Validation strategy A validation strategy developed by Scheepers-Hoeks et al.,14 which we modified, was used. The development and validation strategy developed by Scheepers-Hoeks et al.14 consists of four steps. The first step is a retrospective technical validation to confirm that the used CDSS parameters are correctly linked to the data in the EHR. During the second step, all alerts are assessed for clinical relevance, actionability and usefulness by an expert team. In step 3, the CDSS is adjusted to assure prospective correct alerts in daily clinical practice. The fourth step is to optimize suitability of the CDSS in practice, which is done by continuous technical and therapeutic maintenance after implementation. In our study, the fourth step was left out. Technical validation The technical validation of the CDSS algorithm was performed to check whether the CDSS algorithm creates technically valid definitions. This was done by assessing if the parameters in the algorithm were correctly linked to the correct parameters in our EHR; e.g. for the algorithm rule ‘systolic blood pressure should be stable without inotropics or fluid resuscitation’, it was checked if the correct EHR data (the systolic blood pressure in our EHR for this patient on the day of assessment by the CDSS) were transferred to the algorithm for a correct interpretation of the systolic blood pressure. We also checked for this algorithm rule whether the correct inotropics or fluid resuscitation were transferred and if alerts were suppressed in case of a predefined dose (see Table S2 for these inotropics and fluid resuscitation and doses). For the algorithm rule ‘temperature below 38.1°C for a duration of 24 h (notification when antipyretics are used)’ and ‘temperature above 36°C for a duration of 24 h’, it was checked if the correct temperature from our EHR was transferred to the algorithm and if the duration of 24 h was correctly applied by the algorithm rule. We also checked if the correct antipyretics were transferred and if a notification was generated if the daily dose exceeded a predefined dose (see Table S1). For the algorithm rule ‘patient uses oral medication or no systemic medication at all’ we checked if the data in our EHR regarding the medication of patients and route of administration were correctly transferred to the algorithm. We also checked if the alerts were suppressed in case of total parenteral nutrition use. For the algorithm rule ‘alert suppressed in case of Staphylococcus aureus bacteraemia’ we checked if the data in our EHR regarding S. aureus bacteraemia were correctly transferred. Table 2. Actions following the alertsa in clinical practice Categories Already in consultationb Consultation not needed New consultation Total No action: advice would not change the antibiotic policy 257 134 NA 391 (46.5%) Antibiotic had already been discontinued or switched to oral therapy (clinically relevant) 70 293 1 364 (43.3%) New advice given (clinically relevant and useful) 52 1 32 85 (10.1%) Total 379 428 33 840 (100%) Categories Already in consultationb Consultation not needed New consultation Total No action: advice would not change the antibiotic policy 257 134 NA 391 (46.5%) Antibiotic had already been discontinued or switched to oral therapy (clinically relevant) 70 293 1 364 (43.3%) New advice given (clinically relevant and useful) 52 1 32 85 (10.1%) Total 379 428 33 840 (100%) NA, not applicable. a Alerts were given 84 h after start of iv antibiotics to the Antibiotic Stewardship Team. b Consultation by the ID consultancy service. Table 2. Actions following the alertsa in clinical practice Categories Already in consultationb Consultation not needed New consultation Total No action: advice would not change the antibiotic policy 257 134 NA 391 (46.5%) Antibiotic had already been discontinued or switched to oral therapy (clinically relevant) 70 293 1 364 (43.3%) New advice given (clinically relevant and useful) 52 1 32 85 (10.1%) Total 379 428 33 840 (100%) Categories Already in consultationb Consultation not needed New consultation Total No action: advice would not change the antibiotic policy 257 134 NA 391 (46.5%) Antibiotic had already been discontinued or switched to oral therapy (clinically relevant) 70 293 1 364 (43.3%) New advice given (clinically relevant and useful) 52 1 32 85 (10.1%) Total 379 428 33 840 (100%) NA, not applicable. a Alerts were given 84 h after start of iv antibiotics to the Antibiotic Stewardship Team. b Consultation by the ID consultancy service. Retrospective clinical validation All alerts generated by the IVOS algorithm were assessed for clinical relevance, actionability and usefulness. An alert was clinically relevant if the patient was able to switch from iv to oral antibiotic therapy (fulfilled Delphi switch criteria A–E) (Table 1). By chart review, two reviewers (H. A. and Tanca Minderhoud) determined which patients should have been switched, using the IVOS criteria mentioned in Table 1 as the gold standard. Discrepancies between the generated alerts and the gold standard were analysed, discussed with a third reviewer (A. V.) and if needed small modifications to the CDSS algorithm were made (e.g. no suppression of alert, but a notification, when antipyretics are used). Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were used to express the quality of the CDSS algorithm, using the consensus-based and operationalized IVOS criteria (Table 1) as the gold standard. The sensitivity was calculated by dividing the number of patients that were able to switch from iv to oral antibiotic therapy (fulfilled Delphi switch criteria A–E) (Table 1) identified by the IVOS algorithm (the true positives) by the total number of patients that were able to switch from iv to oral antibiotic therapy. The specificity was calculated by dividing the number of patients that were not able to switch from iv to oral antibiotic therapy for which no alert was generated (the true negatives) by the IVOS algorithm by the total number of patients that were not able to switch from iv to oral antibiotic therapy. The PPV was calculated by dividing the number of true positives by the total number of alerts. The NPV was calculated by dividing the number of true negatives by the total number of patients without alerts. An alert was actionable if an oral variant of the antibiotic with good bioavailability existed, including other antibiotics with appropriate microbiological profile (fulfilled Delphi switch criteria A–F) (Table 1). Alerts were useful if they were both clinically relevant and actionable. We also assessed how many patients for which the alerts were useful had been switched to oral antibiotic therapy within 24 h after fulfilling the Delphi switch criteria for the first time. Prospective evaluation of usefulness in daily practice We then implemented the CDSS algorithm in daily clinical practice, linking the algorithm to the database of our EHR. Each day the CDSS algorithm generated a report with patients fulfilling the switch criteria. This report was automatically sent to the ID specialist of the Antibiotic Stewardship Team. The ID specialist then assessed whether the patient could switch to oral therapy and contacted the treating physician if this was possible. The ID specialist noted the performed action in the system by categorizing the alert in the system. The categories that could be selected were: (i) no action; (ii) antibiotic had already been discontinued or switched to oral therapy; and (iii) new advice given. The main category ‘no action’ includes: advice would not change policy; patient died; and not possible yet to switch to oral antibiotic. This category was considered not clinically relevant. The second main category includes the following categories: patient will soon be or has just been switched to oral antibiotic; and antibiotic discontinued or discontinuation scheduled at short notice. This category was considered clinically relevant. The main category ‘new advice given’ includes the following categories: new advice given; advice to discontinue antibiotic; advice to switch from iv to oral antibiotic; and advice to change antibiotic to a smaller spectrum. This category was considered clinically relevant and useful. We then assessed the clinical relevance and usefulness of the CDSS algorithm in clinical practice during the implementation period January–April 2017. Results Technical validation The retrospective study population consisted of 200 adult hospitalized patients using antibiotics during the 2 week study period. The median age of these patients was 61 years (IQR: 26) and 45% were female. The parameters in the CDSS algorithm were linked to the correct parameters in the EHR in 99.5% of the cases. One case was considered technically incorrect because the algorithm identified the patient as a suitable candidate for switching while fluid resuscitation was being prescribed. The NaCl infusion 1000 mL formulation used for resuscitation was not included in the original CDSS algorithm. This was included in the CDSS redesign. Retrospective clinical validation The algorithm generated a switch alert for 72 of the 73 patients that could have been switched according to the consensus-based switch criteria A–E9 (Table 1), resulting in a sensitivity of 98.6%. Of the 127 patients that should not be switched according to the gold standard, the CDSS algorithm generated an alert in 22 patients, resulting in a specificity of 82.7%. The false positive alerts were mainly caused by presence of an infection considered a contra-indication for switching, such as central nervous system infection, endocarditis, empyema or sepsis. These patients could not be identified with the CDSS algorithm, because criterion E could not be translated into a computer interpretable format in this EHR. Overall, the CDSS algorithm had an NPV of 99.1% and a PPV of 76.6%. Sixty of the 73 patients eligible for IVOS received an antibiotic for which an oral variant was available. Ten patients could not be switched at the moment of the alert because no oral variant, including other antibiotics with appropriate microbiological profile, were available. These patients could not be identified with the CDSS algorithm, because criterion F could not be translated into a computer interpretable format. Three patients could not be switched because the gastrointestinal tract was not functioning properly; these patients were not identified with IVOS algorithm criterion C (Table 1). Thus, 73 of the 200 patients should have been switched to oral antibiotic therapy based on the Delphi criteria and for 60 of these patients a suitable oral alternative antibiotic was available. Eighty-five per cent (51 of 60) of the patients that could have been switched to oral antibiotic therapy were not switched within 24 h after fulfilling the Delphi switch criteria for the first time. Prospective evaluation of usefulness in daily practice The CDSS algorithm generated 840 IVOS alerts for 840 prescriptions in 773 different patients during the period January–April 2017. In 379 of the alerts (45%), the ID specialist had already been consulted prior to the alert. For 391 prescriptions (46.5%) an alert was generated that would not change the antibiotic policy, e.g. because of presence of sepsis (Table 2). These patients with sepsis could not be identified with the CDSS algorithm, because criterion E could not be translated into a computer interpretable format. A total of 449 (364 + 85) alerts (53.5%) were clinically relevant. In 364 of these clinically relevant alerts, the iv antibiotic had just been discontinued or switched to oral therapy. Despite a very active ID consultancy service and the (relatively late) generation of an alert, 84 h after the start of therapy, the IVOS alert resulted in advice to change the antibiotic policy in 85 (10.1%) prescriptions. These advices varied from switching to stopping or changing antibiotics. Discussion In this study, we validated a CDSS algorithm, which generates reports with IVOS candidates directed to the ID specialist of the Antibiotic Stewardship Team and assessed its usefulness in daily clinical practice. In the retrospective validation cohort (n = 200), the CDSS algorithm had a sensitivity of 98.6% and a specificity of 82.7% compared with the gold standard of the Delphi switch criteria.9 This resulted in a PPV of 76.6% and an NPV of 99.1%. These results indicate that the CDSS algorithm is a valid instrument to identify IVOS candidates. In the prospective clinical cohort, 85 of 840 alerts (10.1%) were considered both clinically relevant and useful, and resulted in antibiotic advice. Of the 840 alerts, 364 (43.4%) were clinically relevant, but antibiotics were already switched or discontinued by the treating physician or this was scheduled at short notice. This may be explained by the fact that the CDSS alerts were generated 84 h after the start of the iv antibiotics. This relatively long period was chosen to evaluate the contribution of the CDSS on top of usual (switch) care. With a shorter interval to the alert or in a setting with a less active ID service, these clinically relevant alerts might also be potentially useful. The use of a CDSS to facilitate appropriate antibiotic use, such as timely IVOS, is being recommended.15 Various other interventions to facilitate a timely switch have been assessed, such as education and the introduction of IVOS guidelines.6 These strategies have shown to be successful in decreasing the use of iv antibiotics. CDSSs have the advantage of not being dependent on the attending physician. Because misconceptions about the efficacy of oral antibiotic treatment can lead to unnecessary prolonged use of iv antibiotics,16,17 educational strategies remain important. Educational messages could be incorporated in to the CDSS or communicated by the ID specialist. Our CDSS is the first IVOS CDSS based on IVOS criteria for which consensus was reached within an international, qualified selected expert panel. A standardized identification of IVOS candidates is valuable, because there exists a wide variation in the criteria that physicians use to determine if a patient is able to switch to oral antibiotics.8,18,19 Other reported IVOS CDSS algorithms used local criteria20,21 or used very general rules, such as a certain duration of iv therapy or/and an active order for scheduled oral medications or an oral diet.18,22 Of the 840 alerts, 391 (46.5%) generated by our CDSS were not clinically relevant and therefore had no consequences for the antibiotic policy. An explanation for this is that not all Delphi IVOS switch criteria could be translated into the CDSS, because only coded or numerical data can be effectively used in a CDSS format. Assessment of criteria such as presence of an intravascular focus or sepsis, which could not be translated into the CDSS, was left to the discretion of the ID specialist. Because over-alerting may cause alert fatigue, further improvement of the CDSS is important. With advanced coding of data in EHR the efficacy of the CDSS can be improved, thereby also reducing this risk of alert fatigue. To our knowledge only the study of Lammers et al.23 used the same validation strategy to validate their IVOS CDSS algorithm.14 They found a comparable percentage of parameters in their CDSS algorithm, which were linked to the correct parameters in the EHR (98.9%) and in their retrospective study a similar PPV (82.1%) was found. In our retrospective validation cohort, 85% (51 of 60) of the patients that could have been switched to oral antibiotic therapy remained on iv therapy for at least 24 h after fulfilling the switch criteria. This is a higher percentage compared with other studies, reporting that 38.6%–73.9% of patients that were eligible for an early IVOS remained on iv antibiotics longer than necessary.4–6 Possible explanations for this difference are a retrospective versus prospective setting, the used IVOS criteria, time after which IVOS had to take place, implemented IVOS interventions and included departments. Sevinc et al.6 for example prospectively assessed all antibiotic prescriptions for the departments of internal medicine, surgery and pulmonology, whereas we assessed this retrospectively in all departments, except for the cardiothoracic ICU and paediatric and psychiatric wards. This study has strengths and limitations. A strength of this study is that we developed a technically well-validated IVOS CDSS, which is based on IVOS criteria for which consensus was reached within an international, qualified expert panel using a Delphi procedure. Another strength is that we assessed the usefulness of this CDSS in daily clinical practice for a large group of patients. The study has also some limitations that should be taken into account when interpreting the results. Cases categorized as ‘advice would not change the antibiotic policy’ may also include cases in which earlier advice was given by the ID specialist to discontinue or switch the iv antibiotic. These alerts were also correct, but not classified as clinically relevant. Therefore, the potential usefulness of the IVOS CDSS is probably higher in a setting with a less active ID consultancy service. Since in 379 of the alerts (45%) the ID specialist had already been consulted prior to the alert, we expect that the CDSS would be more effective in facilitating antimicrobial stewardship teams in hospitals without an active ID consultancy service. However, it should be noted that the active ID consultancy service in our hospital provides an additional step in patient safety. It is important to be aware of possible unsafe situations, e.g. when an alert is generated for a patient with an infection for which a switch is contra-indicated. This is possible in a hospital setting without coding of the diagnosis of the patient in the EHR, such as in our EHR. Of great importance is that these decision support systems should always be seen as an aid, with the definite decision still being at the physician’s discretion. Moreover, a setting without an active ID consultancy service will have an influence on the NPV and PPV. Further research of the CDSS should therefore include non-academic hospitals without an active ID consultancy system to assess the safety, NPV, PPV and usefulness in these settings. In conclusion, this study shows that a CDSS based on IVOS criteria developed with a Delphi procedure is effective in helping antimicrobial stewardship teams to select candidates for IVOS. Moreover, even in a hospital with an active ID consultancy service this CDSS had additional value to increase IVOS rates. In an era of increasing use of EHR, this IVOS CDSS has the potential to improve the quality of antibiotic use. With further coding of data in EHR the efficacy of the IVOS CDSS can be improved. Acknowledgements We thank Tanca Minderhoud for her cooperation in this study. We also thank Arjen van Vliet, Bart Dannis, Michiel Smet and Marius Vogel who helped develop the CDSS algorithm. Our thanks also go to the ID consultants for categorizing the alerts during the implementation period. Funding This work was supported by the Netherlands Organization for Health Research and Development (ZonMW) (grant number 836021021). Transparency declarations None to declare. Supplementary data Tables S1 and S2 are available as Supplementary data at JAC Online. References 1 Schuts EC , Hulscher ME , Mouton JW et al. Current evidence on hospital antimicrobial stewardship objectives: a systematic review and meta-analysis . Lancet Infect Dis 2016 ; 16 : 847 – 56 . Google Scholar CrossRef Search ADS PubMed 2 Ramirez JA , Vargas S , Ritter GW et al. Early switch from intravenous to oral antibiotics and early hospital discharge: a prospective observational study of 200 consecutive patients with community-acquired pneumonia . Arch Intern Med 1999 ; 159 : 2449 – 54 . Google Scholar CrossRef Search ADS PubMed 3 Laing RB , Mackenzie AR , Shaw H et al. The effect of intravenous-to-oral switch guidelines on the use of parenteral antimicrobials in medical wards . J Antimicrob Chemother 1998 ; 42 : 107 – 11 . Google Scholar CrossRef Search ADS PubMed 4 Mertz D , Koller M , Haller P et al. Outcomes of early switching from intravenous to oral antibiotics on medical wards . J Antimicrob Chemother 2009 ; 64 : 188 – 99 . Google Scholar CrossRef Search ADS PubMed 5 Shrayteh ZM , Rahal MK , Malaeb DN. Practice of switch from intravenous to oral antibiotics . SpringerPlus 2014 ; 3 : 717 . Google Scholar CrossRef Search ADS PubMed 6 Sevinc F , Prins JM , Koopmans RP et al. Early switch from intravenous to oral antibiotics: guidelines and implementation in a large teaching hospital . J Antimicrob Chemother 1999 ; 43 : 601 – 6 . Google Scholar CrossRef Search ADS PubMed 7 Duncan RA. Controlling use of antimicrobial agents . Infect Control Hosp Epidemiol 1997 ; 18 : 260 – 6 . Google Scholar CrossRef Search ADS PubMed 8 Nathwani D , Lawson W , Dryden M et al. Implementing criteria-based early switch/early discharge programmes: a European perspective . Clin Microbiol Infect 2015 ; 21 Suppl 2: S47 – 55 . Google Scholar CrossRef Search ADS PubMed 9 Akhloufi H , Hulscher M , Melles DC et al. Development of operationalized intravenous to oral antibiotic switch criteria . J Antimicrob Chemother 2017 ; 72 : 543 – 6 . Google Scholar CrossRef Search ADS PubMed 10 Charani E , Cooke J , Holmes A. Antibiotic stewardship programmes–what’s missing? J Antimicrob Chemother 2010 ; 65 : 2275 – 7 . Google Scholar CrossRef Search ADS PubMed 11 Lau BD , Pinto BL , Thiemann DR et al. Budget impact analysis of conversion from intravenous to oral medication when clinically eligible for oral intake . Clin Ther 2011 ; 33 : 1792 – 6 . Google Scholar CrossRef Search ADS PubMed 12 Goff DA , Bauer KA , Reed EE et al. Is the ‘low-hanging fruit’ worth picking for antimicrobial stewardship programs? Clin Infect Dis 2012 ; 55 : 587 – 92 . Google Scholar CrossRef Search ADS PubMed 13 Akhloufi H , Streefkerk RH , Melles DC et al. Point prevalence of appropriate antimicrobial therapy in a Dutch university hospital . Eur J Clin Microbiol Infect Dis 2015 ; 34 : 1631 – 7 . Google Scholar CrossRef Search ADS PubMed 14 Scheepers-Hoeks AM , Grouls RJ , Neef C et al. Strategy for implementation and first results of advanced clinical decision support in hospital pharmacy practice . Stud Health Technol Inform 2009 ; 148 : 142 – 8 . Google Scholar PubMed 15 Dellit TH , Owens RC , McGowan JE et al. Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America guidelines for developing an institutional program to enhance antimicrobial stewardship . Clin Infect Dis 2007 ; 44 : 159 – 77 . Google Scholar CrossRef Search ADS PubMed 16 Vanstraelen K , Verhaegen J , Peetermans WE et al. Stimulation of the i.v. to oral switch of bioavailable drugs by phone calls in a Belgian tertiary care hospital . Acta Clin Belg 2013 ; 68 : 179 – 82 . Google Scholar CrossRef Search ADS PubMed 17 Engel MF , Postma DF , Hulscher ME et al. Barriers to an early switch from intravenous to oral antibiotic therapy in hospitalised patients with CAP . Eur Respir J 2013 ; 41 : 123 – 30 . Google Scholar CrossRef Search ADS PubMed 18 Fischer MA , Solomon DH , Teich JM et al. Conversion from intravenous to oral medications: assessment of a computerized intervention for hospitalized patients . Arch Intern Med 2003 ; 163 : 2585 – 9 . Google Scholar CrossRef Search ADS PubMed 19 Halm EA , Switzer GE , Mittman BS et al. What factors influence physicians’ decisions to switch from intravenous to oral antibiotics for community-acquired pneumonia? J Gen Intern Med 2001 ; 16 : 599 – 605 . Google Scholar CrossRef Search ADS PubMed 20 Sprong T , Pot H , Dofferhoff T et al. Intelligent electronic trigger tool to optimise intravenous to oral antibiotic switch . Clin Microbiol Infect 2012 ; 18 : 106. 21 Beeler PE , Kuster SP , Eschmann E et al. Earlier switching from intravenous to oral antibiotics owing to electronic reminders . Int J Antimicrob Agents 2015 ; 46 : 428 – 33 . Google Scholar CrossRef Search ADS PubMed 22 Prins JM , Nellen JFJB , Koopmans RP et al. Electronic drug ordering system can be helpful to implement iv-oral switch guidelines . J Antimicrob Chemother 2000 ; 46 : 518 – 9 . Google Scholar CrossRef Search ADS PubMed 23 Lammers HJW , Wasylewicz ATM , Scheepers-Hoeks AMJW et al. Effect van een beslisregel-gestuurde interventie op vroege omzetting van intraveneuze naar orale antibiotica . PW Wetenschappelijk Platform 2013 ; 7 : a1311 . © The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Antimicrobial Chemotherapy. All rights reserved. For permissions, please email: [email protected]. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Antimicrobial Chemotherapy Oxford University Press

A clinical decision support system algorithm for intravenous to oral antibiotic switch therapy: validity, clinical relevance and usefulness in a three-step evaluation study

Journal of Antimicrobial Chemotherapy , Volume Advance Article (8) – Apr 26, 2018
Download PDF
6 pages

Loading next page...
 
/lp/ou_press/a-clinical-decision-support-system-algorithm-for-intravenous-to-oral-uq0jWxJWJ1

References (24)

Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Antimicrobial Chemotherapy. All rights reserved. For permissions, please email: [email protected].
ISSN
0305-7453
eISSN
1460-2091
DOI
10.1093/jac/dky141
Publisher site
See Article on Publisher Site

Abstract

Abstract Objectives To evaluate a clinical decision support system (CDSS) based on consensus-based intravenous to oral switch criteria, which identifies intravenous to oral switch candidates. Methods A three-step evaluation study of a stand-alone CDSS with electronic health record interoperability was performed at the Erasmus University Medical Centre in the Netherlands. During the first step, we performed a technical validation. During the second step, we determined the sensitivity, specificity, negative predictive value and positive predictive value in a retrospective cohort of all hospitalized adult patients starting at least one therapeutic antibacterial drug between 1 and 16 May 2013. ICU, paediatric and psychiatric wards were excluded. During the last step the clinical relevance and usefulness was prospectively assessed by reports to infectious disease specialists. An alert was considered clinically relevant if antibiotics could be discontinued or switched to oral therapy at the time of the alert. Results During the first step, one technical error was found. The second step yielded a positive predictive value of 76.6% and a negative predictive value of 99.1%. The third step showed that alerts were clinically relevant in 53.5% of patients. For 43.4% it had already been decided to discontinue or switch the intravenous antibiotics by the treating physician. In 10.1%, the alert resulted in advice to change antibiotic policy and was considered useful. Conclusions This prospective cohort study shows that the alerts were clinically relevant in >50% (n = 449) and useful in 10% (n = 85). The CDSS needs to be evaluated in hospitals with varying activity of infectious disease consultancy services as this probably influences usefulness. Introduction One of the most cost-effective and safe objectives of a hospital antimicrobial stewardship programme is the timely switch from intravenous (iv) to oral antibiotic therapy.1 A timely iv to oral switch (IVOS) has many advantages, such as reduced incidence of catheter-related infections and a decreased hospital length of stay.1–3 However, up to two-thirds of patients eligible for an IVOS remain on iv antibiotics longer than necessary.4–6 A possible explanation is that reassessment of iv-started antibiotics is often not done, for instance owing to time constraints and change of staff.7 In addition, different IVOS criteria are being used to determine whether a patient can be switched from iv to oral antibiotic therapy.8 To address the latter issue, an international Delphi procedure was recently performed to reach consensus among international experts on a set of IVOS criteria that have to be met in adult hospitalized patients for a safe IVOS after 48–72 h of iv therapy.9 This resulted in the development of six IVOS switch criteria, operationalized by 16 measurable conditions; e.g. the IVOS switch criterion ‘signs and symptoms related to the infection have to be resolved or improved’ is operationalized by the following two measurable conditions: temperature should be <38.3°C without antipyretics and >36°C (Table 1). Table 1. Clinical rule iv oral antibiotic switch based on operationalized and consensus-based switch criteria9 iv oral antibiotic switch criteria IVOS algorithm A. Vital signs should be good or improving when bad  systolic blood pressure should be stable without inotropics or fluid resuscitation systolic blood pressure >90 mmHg (alert suppressed if inotropics or fluid resuscitation is used) B. Signs and symptoms related to the infection have to be resolved or improved  temperature should be <38.3°C without antipyretics temperature <38.1°C for a duration of 24 h (notification when antipyretics are used)  temperature should be >36°C temperature >36°C for a duration of 24 h C. Gastrointestinal tract has to be intact and functioning patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  absence of the following conditions   malabsorption syndrome   short bowel syndrome   severe gastroparesis   ileus   continuous nasogastric suction D. Oral route should not be compromised patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  no vomiting  patient should be cooperative E. Absence of contra-indicated infections not possible to translate into a computer interpretable formata  adequate antimicrobial concentrations are not achievable at the site of infection by oral administration  absence of the following infections   (severe) sepsis   fasciitis necroticans   CNS infection   Staphylococcus aureus bacteraemia alert suppressed in case of S. aureus bacteraemia   endovascular infection (e.g. endocarditis) F. An oral variant of the antibiotic with good bioavailability has to exist, including other antibiotics with appropriate microbiological profile not possible to translate into a computer interpretable formata iv oral antibiotic switch criteria IVOS algorithm A. Vital signs should be good or improving when bad  systolic blood pressure should be stable without inotropics or fluid resuscitation systolic blood pressure >90 mmHg (alert suppressed if inotropics or fluid resuscitation is used) B. Signs and symptoms related to the infection have to be resolved or improved  temperature should be <38.3°C without antipyretics temperature <38.1°C for a duration of 24 h (notification when antipyretics are used)  temperature should be >36°C temperature >36°C for a duration of 24 h C. Gastrointestinal tract has to be intact and functioning patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  absence of the following conditions   malabsorption syndrome   short bowel syndrome   severe gastroparesis   ileus   continuous nasogastric suction D. Oral route should not be compromised patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  no vomiting  patient should be cooperative E. Absence of contra-indicated infections not possible to translate into a computer interpretable formata  adequate antimicrobial concentrations are not achievable at the site of infection by oral administration  absence of the following infections   (severe) sepsis   fasciitis necroticans   CNS infection   Staphylococcus aureus bacteraemia alert suppressed in case of S. aureus bacteraemia   endovascular infection (e.g. endocarditis) F. An oral variant of the antibiotic with good bioavailability has to exist, including other antibiotics with appropriate microbiological profile not possible to translate into a computer interpretable formata a Except for S. aureus bacteraemia, which was assessed by the algorithm. Assessment of criteria that could not be translated into a computer interpretable format was left to the discretion of the ID specialist. Table 1. Clinical rule iv oral antibiotic switch based on operationalized and consensus-based switch criteria9 iv oral antibiotic switch criteria IVOS algorithm A. Vital signs should be good or improving when bad  systolic blood pressure should be stable without inotropics or fluid resuscitation systolic blood pressure >90 mmHg (alert suppressed if inotropics or fluid resuscitation is used) B. Signs and symptoms related to the infection have to be resolved or improved  temperature should be <38.3°C without antipyretics temperature <38.1°C for a duration of 24 h (notification when antipyretics are used)  temperature should be >36°C temperature >36°C for a duration of 24 h C. Gastrointestinal tract has to be intact and functioning patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  absence of the following conditions   malabsorption syndrome   short bowel syndrome   severe gastroparesis   ileus   continuous nasogastric suction D. Oral route should not be compromised patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  no vomiting  patient should be cooperative E. Absence of contra-indicated infections not possible to translate into a computer interpretable formata  adequate antimicrobial concentrations are not achievable at the site of infection by oral administration  absence of the following infections   (severe) sepsis   fasciitis necroticans   CNS infection   Staphylococcus aureus bacteraemia alert suppressed in case of S. aureus bacteraemia   endovascular infection (e.g. endocarditis) F. An oral variant of the antibiotic with good bioavailability has to exist, including other antibiotics with appropriate microbiological profile not possible to translate into a computer interpretable formata iv oral antibiotic switch criteria IVOS algorithm A. Vital signs should be good or improving when bad  systolic blood pressure should be stable without inotropics or fluid resuscitation systolic blood pressure >90 mmHg (alert suppressed if inotropics or fluid resuscitation is used) B. Signs and symptoms related to the infection have to be resolved or improved  temperature should be <38.3°C without antipyretics temperature <38.1°C for a duration of 24 h (notification when antipyretics are used)  temperature should be >36°C temperature >36°C for a duration of 24 h C. Gastrointestinal tract has to be intact and functioning patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  absence of the following conditions   malabsorption syndrome   short bowel syndrome   severe gastroparesis   ileus   continuous nasogastric suction D. Oral route should not be compromised patient uses oral medication or no systemic medication at all (alert suppressed in case of total parenteral nutrition use)  no vomiting  patient should be cooperative E. Absence of contra-indicated infections not possible to translate into a computer interpretable formata  adequate antimicrobial concentrations are not achievable at the site of infection by oral administration  absence of the following infections   (severe) sepsis   fasciitis necroticans   CNS infection   Staphylococcus aureus bacteraemia alert suppressed in case of S. aureus bacteraemia   endovascular infection (e.g. endocarditis) F. An oral variant of the antibiotic with good bioavailability has to exist, including other antibiotics with appropriate microbiological profile not possible to translate into a computer interpretable formata a Except for S. aureus bacteraemia, which was assessed by the algorithm. Assessment of criteria that could not be translated into a computer interpretable format was left to the discretion of the ID specialist. These operationalized consensus criteria are a first step towards standardized IVOS criteria. However, to improve appropriate antibiotic use in an effective and sustainable manner, more is needed than guidance and instructions.10 A promising stewardship intervention to facilitate a timely IVOS is a clinical decision support system (CDSS) that automatically generates reminding alerts.11,12 Earlier studies on CDSSs have shown that an important pitfall, which can impede the success of a CDSS, is alert fatigue. To prevent alert fatigue a well-designed CDSS with high specificity and sensitivity is crucial. We developed such a CDSS algorithm that is based on the operationalized consensus criteria for IVOS.9 In this study, we validated this CDSS algorithm and assessed its clinical relevance and usefulness in daily clinical practice. Methods Setting This study was conducted at the Erasmus University Medical Centre (Erasmus MC) in Rotterdam in the Netherlands, a 1237 bed tertiary care centre. The Erasmus MC uses an electronic health record (EHR) with integrated computerized prescriber order entry. This system integrates patient data, medication prescriptions, laboratory data, surgical reports and radiology reports. The Department of Medical Microbiology and Infectious Diseases of this hospital provides an active infectious disease (ID) consultancy service, in which ID consultants actively give the attending physicians antibiotic advice. They are also being consulted for antibiotic advice in nearly half of the patients using antibiotic therapy.13 Study population The study population for the technical and the retrospective clinical validation was extracted from the computerized prescriber order entry and consisted of all hospitalized adult patients starting at least one antibacterial drug (ATC code J01) between 1 and 16 May 2013. Adult ICU patients were included and patients on the cardiothoracic ICU and the paediatric and psychiatric wards were excluded. Patients using prophylactic antibiotics were also excluded. Co-trimoxazole at a maximum daily dose of 480 mg and cefazolin given once were always considered prophylaxis. Antibiotics given according to a local prophylaxis protocol were also defined as prophylaxis. This was assessed manually. The same criteria were used for the prospective study. IVOS algorithm A multidisciplinary team, consisting of an ID specialist, clinical microbiologists, hospital pharmacist experienced in decision support, information technology team and a researcher, developed the CDSS algorithm. The consensus-based and operationalized criteria9 were translated into a computer-interpretable format. All adult hospitalized patients with at least one iv antibiotic prescription for a duration of at least 84 h were selected by the algorithm. This relatively long time period was chosen because we wanted to evaluate the contribution of the CDSS on top of usual (switch) care. An alert indicating that the patient could be safely switched was generated when the conditions described in Table 1 were met. A report with all patients with an alert was automatically generated on a daily basis and directed to the ID specialist of the Antibiotic Stewardship Team. Criterion E ‘Absence of contra-indicated infections’ could not be translated into the CDSS algorithm. The same applied to criterion F ‘An oral variant of the antibiotic with good bioavailability has to exist, including other antibiotics with appropriate microbiological profile’. These criteria were left to the discretion of the ID specialist. An overview of the included antipyretics (used to assess criterion B), inotropics and fluid resuscitation (used to assess criterion A) can be found in Table S1 (available as Supplementary data at JAC Online). Validation strategy A validation strategy developed by Scheepers-Hoeks et al.,14 which we modified, was used. The development and validation strategy developed by Scheepers-Hoeks et al.14 consists of four steps. The first step is a retrospective technical validation to confirm that the used CDSS parameters are correctly linked to the data in the EHR. During the second step, all alerts are assessed for clinical relevance, actionability and usefulness by an expert team. In step 3, the CDSS is adjusted to assure prospective correct alerts in daily clinical practice. The fourth step is to optimize suitability of the CDSS in practice, which is done by continuous technical and therapeutic maintenance after implementation. In our study, the fourth step was left out. Technical validation The technical validation of the CDSS algorithm was performed to check whether the CDSS algorithm creates technically valid definitions. This was done by assessing if the parameters in the algorithm were correctly linked to the correct parameters in our EHR; e.g. for the algorithm rule ‘systolic blood pressure should be stable without inotropics or fluid resuscitation’, it was checked if the correct EHR data (the systolic blood pressure in our EHR for this patient on the day of assessment by the CDSS) were transferred to the algorithm for a correct interpretation of the systolic blood pressure. We also checked for this algorithm rule whether the correct inotropics or fluid resuscitation were transferred and if alerts were suppressed in case of a predefined dose (see Table S2 for these inotropics and fluid resuscitation and doses). For the algorithm rule ‘temperature below 38.1°C for a duration of 24 h (notification when antipyretics are used)’ and ‘temperature above 36°C for a duration of 24 h’, it was checked if the correct temperature from our EHR was transferred to the algorithm and if the duration of 24 h was correctly applied by the algorithm rule. We also checked if the correct antipyretics were transferred and if a notification was generated if the daily dose exceeded a predefined dose (see Table S1). For the algorithm rule ‘patient uses oral medication or no systemic medication at all’ we checked if the data in our EHR regarding the medication of patients and route of administration were correctly transferred to the algorithm. We also checked if the alerts were suppressed in case of total parenteral nutrition use. For the algorithm rule ‘alert suppressed in case of Staphylococcus aureus bacteraemia’ we checked if the data in our EHR regarding S. aureus bacteraemia were correctly transferred. Table 2. Actions following the alertsa in clinical practice Categories Already in consultationb Consultation not needed New consultation Total No action: advice would not change the antibiotic policy 257 134 NA 391 (46.5%) Antibiotic had already been discontinued or switched to oral therapy (clinically relevant) 70 293 1 364 (43.3%) New advice given (clinically relevant and useful) 52 1 32 85 (10.1%) Total 379 428 33 840 (100%) Categories Already in consultationb Consultation not needed New consultation Total No action: advice would not change the antibiotic policy 257 134 NA 391 (46.5%) Antibiotic had already been discontinued or switched to oral therapy (clinically relevant) 70 293 1 364 (43.3%) New advice given (clinically relevant and useful) 52 1 32 85 (10.1%) Total 379 428 33 840 (100%) NA, not applicable. a Alerts were given 84 h after start of iv antibiotics to the Antibiotic Stewardship Team. b Consultation by the ID consultancy service. Table 2. Actions following the alertsa in clinical practice Categories Already in consultationb Consultation not needed New consultation Total No action: advice would not change the antibiotic policy 257 134 NA 391 (46.5%) Antibiotic had already been discontinued or switched to oral therapy (clinically relevant) 70 293 1 364 (43.3%) New advice given (clinically relevant and useful) 52 1 32 85 (10.1%) Total 379 428 33 840 (100%) Categories Already in consultationb Consultation not needed New consultation Total No action: advice would not change the antibiotic policy 257 134 NA 391 (46.5%) Antibiotic had already been discontinued or switched to oral therapy (clinically relevant) 70 293 1 364 (43.3%) New advice given (clinically relevant and useful) 52 1 32 85 (10.1%) Total 379 428 33 840 (100%) NA, not applicable. a Alerts were given 84 h after start of iv antibiotics to the Antibiotic Stewardship Team. b Consultation by the ID consultancy service. Retrospective clinical validation All alerts generated by the IVOS algorithm were assessed for clinical relevance, actionability and usefulness. An alert was clinically relevant if the patient was able to switch from iv to oral antibiotic therapy (fulfilled Delphi switch criteria A–E) (Table 1). By chart review, two reviewers (H. A. and Tanca Minderhoud) determined which patients should have been switched, using the IVOS criteria mentioned in Table 1 as the gold standard. Discrepancies between the generated alerts and the gold standard were analysed, discussed with a third reviewer (A. V.) and if needed small modifications to the CDSS algorithm were made (e.g. no suppression of alert, but a notification, when antipyretics are used). Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were used to express the quality of the CDSS algorithm, using the consensus-based and operationalized IVOS criteria (Table 1) as the gold standard. The sensitivity was calculated by dividing the number of patients that were able to switch from iv to oral antibiotic therapy (fulfilled Delphi switch criteria A–E) (Table 1) identified by the IVOS algorithm (the true positives) by the total number of patients that were able to switch from iv to oral antibiotic therapy. The specificity was calculated by dividing the number of patients that were not able to switch from iv to oral antibiotic therapy for which no alert was generated (the true negatives) by the IVOS algorithm by the total number of patients that were not able to switch from iv to oral antibiotic therapy. The PPV was calculated by dividing the number of true positives by the total number of alerts. The NPV was calculated by dividing the number of true negatives by the total number of patients without alerts. An alert was actionable if an oral variant of the antibiotic with good bioavailability existed, including other antibiotics with appropriate microbiological profile (fulfilled Delphi switch criteria A–F) (Table 1). Alerts were useful if they were both clinically relevant and actionable. We also assessed how many patients for which the alerts were useful had been switched to oral antibiotic therapy within 24 h after fulfilling the Delphi switch criteria for the first time. Prospective evaluation of usefulness in daily practice We then implemented the CDSS algorithm in daily clinical practice, linking the algorithm to the database of our EHR. Each day the CDSS algorithm generated a report with patients fulfilling the switch criteria. This report was automatically sent to the ID specialist of the Antibiotic Stewardship Team. The ID specialist then assessed whether the patient could switch to oral therapy and contacted the treating physician if this was possible. The ID specialist noted the performed action in the system by categorizing the alert in the system. The categories that could be selected were: (i) no action; (ii) antibiotic had already been discontinued or switched to oral therapy; and (iii) new advice given. The main category ‘no action’ includes: advice would not change policy; patient died; and not possible yet to switch to oral antibiotic. This category was considered not clinically relevant. The second main category includes the following categories: patient will soon be or has just been switched to oral antibiotic; and antibiotic discontinued or discontinuation scheduled at short notice. This category was considered clinically relevant. The main category ‘new advice given’ includes the following categories: new advice given; advice to discontinue antibiotic; advice to switch from iv to oral antibiotic; and advice to change antibiotic to a smaller spectrum. This category was considered clinically relevant and useful. We then assessed the clinical relevance and usefulness of the CDSS algorithm in clinical practice during the implementation period January–April 2017. Results Technical validation The retrospective study population consisted of 200 adult hospitalized patients using antibiotics during the 2 week study period. The median age of these patients was 61 years (IQR: 26) and 45% were female. The parameters in the CDSS algorithm were linked to the correct parameters in the EHR in 99.5% of the cases. One case was considered technically incorrect because the algorithm identified the patient as a suitable candidate for switching while fluid resuscitation was being prescribed. The NaCl infusion 1000 mL formulation used for resuscitation was not included in the original CDSS algorithm. This was included in the CDSS redesign. Retrospective clinical validation The algorithm generated a switch alert for 72 of the 73 patients that could have been switched according to the consensus-based switch criteria A–E9 (Table 1), resulting in a sensitivity of 98.6%. Of the 127 patients that should not be switched according to the gold standard, the CDSS algorithm generated an alert in 22 patients, resulting in a specificity of 82.7%. The false positive alerts were mainly caused by presence of an infection considered a contra-indication for switching, such as central nervous system infection, endocarditis, empyema or sepsis. These patients could not be identified with the CDSS algorithm, because criterion E could not be translated into a computer interpretable format in this EHR. Overall, the CDSS algorithm had an NPV of 99.1% and a PPV of 76.6%. Sixty of the 73 patients eligible for IVOS received an antibiotic for which an oral variant was available. Ten patients could not be switched at the moment of the alert because no oral variant, including other antibiotics with appropriate microbiological profile, were available. These patients could not be identified with the CDSS algorithm, because criterion F could not be translated into a computer interpretable format. Three patients could not be switched because the gastrointestinal tract was not functioning properly; these patients were not identified with IVOS algorithm criterion C (Table 1). Thus, 73 of the 200 patients should have been switched to oral antibiotic therapy based on the Delphi criteria and for 60 of these patients a suitable oral alternative antibiotic was available. Eighty-five per cent (51 of 60) of the patients that could have been switched to oral antibiotic therapy were not switched within 24 h after fulfilling the Delphi switch criteria for the first time. Prospective evaluation of usefulness in daily practice The CDSS algorithm generated 840 IVOS alerts for 840 prescriptions in 773 different patients during the period January–April 2017. In 379 of the alerts (45%), the ID specialist had already been consulted prior to the alert. For 391 prescriptions (46.5%) an alert was generated that would not change the antibiotic policy, e.g. because of presence of sepsis (Table 2). These patients with sepsis could not be identified with the CDSS algorithm, because criterion E could not be translated into a computer interpretable format. A total of 449 (364 + 85) alerts (53.5%) were clinically relevant. In 364 of these clinically relevant alerts, the iv antibiotic had just been discontinued or switched to oral therapy. Despite a very active ID consultancy service and the (relatively late) generation of an alert, 84 h after the start of therapy, the IVOS alert resulted in advice to change the antibiotic policy in 85 (10.1%) prescriptions. These advices varied from switching to stopping or changing antibiotics. Discussion In this study, we validated a CDSS algorithm, which generates reports with IVOS candidates directed to the ID specialist of the Antibiotic Stewardship Team and assessed its usefulness in daily clinical practice. In the retrospective validation cohort (n = 200), the CDSS algorithm had a sensitivity of 98.6% and a specificity of 82.7% compared with the gold standard of the Delphi switch criteria.9 This resulted in a PPV of 76.6% and an NPV of 99.1%. These results indicate that the CDSS algorithm is a valid instrument to identify IVOS candidates. In the prospective clinical cohort, 85 of 840 alerts (10.1%) were considered both clinically relevant and useful, and resulted in antibiotic advice. Of the 840 alerts, 364 (43.4%) were clinically relevant, but antibiotics were already switched or discontinued by the treating physician or this was scheduled at short notice. This may be explained by the fact that the CDSS alerts were generated 84 h after the start of the iv antibiotics. This relatively long period was chosen to evaluate the contribution of the CDSS on top of usual (switch) care. With a shorter interval to the alert or in a setting with a less active ID service, these clinically relevant alerts might also be potentially useful. The use of a CDSS to facilitate appropriate antibiotic use, such as timely IVOS, is being recommended.15 Various other interventions to facilitate a timely switch have been assessed, such as education and the introduction of IVOS guidelines.6 These strategies have shown to be successful in decreasing the use of iv antibiotics. CDSSs have the advantage of not being dependent on the attending physician. Because misconceptions about the efficacy of oral antibiotic treatment can lead to unnecessary prolonged use of iv antibiotics,16,17 educational strategies remain important. Educational messages could be incorporated in to the CDSS or communicated by the ID specialist. Our CDSS is the first IVOS CDSS based on IVOS criteria for which consensus was reached within an international, qualified selected expert panel. A standardized identification of IVOS candidates is valuable, because there exists a wide variation in the criteria that physicians use to determine if a patient is able to switch to oral antibiotics.8,18,19 Other reported IVOS CDSS algorithms used local criteria20,21 or used very general rules, such as a certain duration of iv therapy or/and an active order for scheduled oral medications or an oral diet.18,22 Of the 840 alerts, 391 (46.5%) generated by our CDSS were not clinically relevant and therefore had no consequences for the antibiotic policy. An explanation for this is that not all Delphi IVOS switch criteria could be translated into the CDSS, because only coded or numerical data can be effectively used in a CDSS format. Assessment of criteria such as presence of an intravascular focus or sepsis, which could not be translated into the CDSS, was left to the discretion of the ID specialist. Because over-alerting may cause alert fatigue, further improvement of the CDSS is important. With advanced coding of data in EHR the efficacy of the CDSS can be improved, thereby also reducing this risk of alert fatigue. To our knowledge only the study of Lammers et al.23 used the same validation strategy to validate their IVOS CDSS algorithm.14 They found a comparable percentage of parameters in their CDSS algorithm, which were linked to the correct parameters in the EHR (98.9%) and in their retrospective study a similar PPV (82.1%) was found. In our retrospective validation cohort, 85% (51 of 60) of the patients that could have been switched to oral antibiotic therapy remained on iv therapy for at least 24 h after fulfilling the switch criteria. This is a higher percentage compared with other studies, reporting that 38.6%–73.9% of patients that were eligible for an early IVOS remained on iv antibiotics longer than necessary.4–6 Possible explanations for this difference are a retrospective versus prospective setting, the used IVOS criteria, time after which IVOS had to take place, implemented IVOS interventions and included departments. Sevinc et al.6 for example prospectively assessed all antibiotic prescriptions for the departments of internal medicine, surgery and pulmonology, whereas we assessed this retrospectively in all departments, except for the cardiothoracic ICU and paediatric and psychiatric wards. This study has strengths and limitations. A strength of this study is that we developed a technically well-validated IVOS CDSS, which is based on IVOS criteria for which consensus was reached within an international, qualified expert panel using a Delphi procedure. Another strength is that we assessed the usefulness of this CDSS in daily clinical practice for a large group of patients. The study has also some limitations that should be taken into account when interpreting the results. Cases categorized as ‘advice would not change the antibiotic policy’ may also include cases in which earlier advice was given by the ID specialist to discontinue or switch the iv antibiotic. These alerts were also correct, but not classified as clinically relevant. Therefore, the potential usefulness of the IVOS CDSS is probably higher in a setting with a less active ID consultancy service. Since in 379 of the alerts (45%) the ID specialist had already been consulted prior to the alert, we expect that the CDSS would be more effective in facilitating antimicrobial stewardship teams in hospitals without an active ID consultancy service. However, it should be noted that the active ID consultancy service in our hospital provides an additional step in patient safety. It is important to be aware of possible unsafe situations, e.g. when an alert is generated for a patient with an infection for which a switch is contra-indicated. This is possible in a hospital setting without coding of the diagnosis of the patient in the EHR, such as in our EHR. Of great importance is that these decision support systems should always be seen as an aid, with the definite decision still being at the physician’s discretion. Moreover, a setting without an active ID consultancy service will have an influence on the NPV and PPV. Further research of the CDSS should therefore include non-academic hospitals without an active ID consultancy system to assess the safety, NPV, PPV and usefulness in these settings. In conclusion, this study shows that a CDSS based on IVOS criteria developed with a Delphi procedure is effective in helping antimicrobial stewardship teams to select candidates for IVOS. Moreover, even in a hospital with an active ID consultancy service this CDSS had additional value to increase IVOS rates. In an era of increasing use of EHR, this IVOS CDSS has the potential to improve the quality of antibiotic use. With further coding of data in EHR the efficacy of the IVOS CDSS can be improved. Acknowledgements We thank Tanca Minderhoud for her cooperation in this study. We also thank Arjen van Vliet, Bart Dannis, Michiel Smet and Marius Vogel who helped develop the CDSS algorithm. Our thanks also go to the ID consultants for categorizing the alerts during the implementation period. Funding This work was supported by the Netherlands Organization for Health Research and Development (ZonMW) (grant number 836021021). Transparency declarations None to declare. Supplementary data Tables S1 and S2 are available as Supplementary data at JAC Online. References 1 Schuts EC , Hulscher ME , Mouton JW et al. Current evidence on hospital antimicrobial stewardship objectives: a systematic review and meta-analysis . Lancet Infect Dis 2016 ; 16 : 847 – 56 . Google Scholar CrossRef Search ADS PubMed 2 Ramirez JA , Vargas S , Ritter GW et al. Early switch from intravenous to oral antibiotics and early hospital discharge: a prospective observational study of 200 consecutive patients with community-acquired pneumonia . Arch Intern Med 1999 ; 159 : 2449 – 54 . Google Scholar CrossRef Search ADS PubMed 3 Laing RB , Mackenzie AR , Shaw H et al. The effect of intravenous-to-oral switch guidelines on the use of parenteral antimicrobials in medical wards . J Antimicrob Chemother 1998 ; 42 : 107 – 11 . Google Scholar CrossRef Search ADS PubMed 4 Mertz D , Koller M , Haller P et al. Outcomes of early switching from intravenous to oral antibiotics on medical wards . J Antimicrob Chemother 2009 ; 64 : 188 – 99 . Google Scholar CrossRef Search ADS PubMed 5 Shrayteh ZM , Rahal MK , Malaeb DN. Practice of switch from intravenous to oral antibiotics . SpringerPlus 2014 ; 3 : 717 . Google Scholar CrossRef Search ADS PubMed 6 Sevinc F , Prins JM , Koopmans RP et al. Early switch from intravenous to oral antibiotics: guidelines and implementation in a large teaching hospital . J Antimicrob Chemother 1999 ; 43 : 601 – 6 . Google Scholar CrossRef Search ADS PubMed 7 Duncan RA. Controlling use of antimicrobial agents . Infect Control Hosp Epidemiol 1997 ; 18 : 260 – 6 . Google Scholar CrossRef Search ADS PubMed 8 Nathwani D , Lawson W , Dryden M et al. Implementing criteria-based early switch/early discharge programmes: a European perspective . Clin Microbiol Infect 2015 ; 21 Suppl 2: S47 – 55 . Google Scholar CrossRef Search ADS PubMed 9 Akhloufi H , Hulscher M , Melles DC et al. Development of operationalized intravenous to oral antibiotic switch criteria . J Antimicrob Chemother 2017 ; 72 : 543 – 6 . Google Scholar CrossRef Search ADS PubMed 10 Charani E , Cooke J , Holmes A. Antibiotic stewardship programmes–what’s missing? J Antimicrob Chemother 2010 ; 65 : 2275 – 7 . Google Scholar CrossRef Search ADS PubMed 11 Lau BD , Pinto BL , Thiemann DR et al. Budget impact analysis of conversion from intravenous to oral medication when clinically eligible for oral intake . Clin Ther 2011 ; 33 : 1792 – 6 . Google Scholar CrossRef Search ADS PubMed 12 Goff DA , Bauer KA , Reed EE et al. Is the ‘low-hanging fruit’ worth picking for antimicrobial stewardship programs? Clin Infect Dis 2012 ; 55 : 587 – 92 . Google Scholar CrossRef Search ADS PubMed 13 Akhloufi H , Streefkerk RH , Melles DC et al. Point prevalence of appropriate antimicrobial therapy in a Dutch university hospital . Eur J Clin Microbiol Infect Dis 2015 ; 34 : 1631 – 7 . Google Scholar CrossRef Search ADS PubMed 14 Scheepers-Hoeks AM , Grouls RJ , Neef C et al. Strategy for implementation and first results of advanced clinical decision support in hospital pharmacy practice . Stud Health Technol Inform 2009 ; 148 : 142 – 8 . Google Scholar PubMed 15 Dellit TH , Owens RC , McGowan JE et al. Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America guidelines for developing an institutional program to enhance antimicrobial stewardship . Clin Infect Dis 2007 ; 44 : 159 – 77 . Google Scholar CrossRef Search ADS PubMed 16 Vanstraelen K , Verhaegen J , Peetermans WE et al. Stimulation of the i.v. to oral switch of bioavailable drugs by phone calls in a Belgian tertiary care hospital . Acta Clin Belg 2013 ; 68 : 179 – 82 . Google Scholar CrossRef Search ADS PubMed 17 Engel MF , Postma DF , Hulscher ME et al. Barriers to an early switch from intravenous to oral antibiotic therapy in hospitalised patients with CAP . Eur Respir J 2013 ; 41 : 123 – 30 . Google Scholar CrossRef Search ADS PubMed 18 Fischer MA , Solomon DH , Teich JM et al. Conversion from intravenous to oral medications: assessment of a computerized intervention for hospitalized patients . Arch Intern Med 2003 ; 163 : 2585 – 9 . Google Scholar CrossRef Search ADS PubMed 19 Halm EA , Switzer GE , Mittman BS et al. What factors influence physicians’ decisions to switch from intravenous to oral antibiotics for community-acquired pneumonia? J Gen Intern Med 2001 ; 16 : 599 – 605 . Google Scholar CrossRef Search ADS PubMed 20 Sprong T , Pot H , Dofferhoff T et al. Intelligent electronic trigger tool to optimise intravenous to oral antibiotic switch . Clin Microbiol Infect 2012 ; 18 : 106. 21 Beeler PE , Kuster SP , Eschmann E et al. Earlier switching from intravenous to oral antibiotics owing to electronic reminders . Int J Antimicrob Agents 2015 ; 46 : 428 – 33 . Google Scholar CrossRef Search ADS PubMed 22 Prins JM , Nellen JFJB , Koopmans RP et al. Electronic drug ordering system can be helpful to implement iv-oral switch guidelines . J Antimicrob Chemother 2000 ; 46 : 518 – 9 . Google Scholar CrossRef Search ADS PubMed 23 Lammers HJW , Wasylewicz ATM , Scheepers-Hoeks AMJW et al. Effect van een beslisregel-gestuurde interventie op vroege omzetting van intraveneuze naar orale antibiotica . PW Wetenschappelijk Platform 2013 ; 7 : a1311 . © The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Antimicrobial Chemotherapy. All rights reserved. For permissions, please email: [email protected]. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

Journal of Antimicrobial Chemotherapy – Oxford University Press

Published: Apr 26, 2018

There are no references for this article.