TY - JOUR
AB - Abstract Background Antimicrobial prescribing behaviour is first established during medical study, but teachers often cite lack of time as an important problem in the implementation of antimicrobial stewardship in the medical curriculum. The use of electronic learning (e-learning) is a potentially time-efficient solution, but its effectiveness in changing long-term prescribing behaviour in medical students is as yet unknown. Methods We performed a prospective controlled intervention study of the long-term effects of a short interactive e-learning course among fourth year medical students in a Dutch university. The e-learning was temporarily implemented as a non-compulsory course during a 6 week period. Six months later, all students underwent an infectious disease-based objective structured clinical examination (OSCE) aimed at simulating postgraduate prescribing. If they passed, each student did the OSCE only once. We created a control group of students from a period when the e-learning was not implemented. Main outcomes were the OSCE pass percentage and knowledge, drug choice and overall scores. We used propensity scores to create equal comparisons. Results We included 71 students in the intervention group and 285 students in the control group. E-learning participation in the intervention group was 81%. The OSCE pass percentage was 86% in the control group versus 97% in the intervention group (+11%, OR 5.9, 95% CI 1.7–20.0). OSCE overall, knowledge and drug choice grades (1–10) were also significantly higher in the intervention group (differences +0.31, +0.31 and +0.51, respectively). Conclusions E-learning during a limited period can significantly improve medical students’ performance of an antimicrobial therapeutic consultation in a situation simulating clinical practice 6 months later. Introduction Education is an essential pillar of antimicrobial stewardship.1 The basis for professional behaviour is laid during the first years of medical study. Interventions to promote prudent antimicrobial prescribing should therefore increasingly focus on undergraduates, rather than on postgraduates only.2 A recent survey revealed that European medical teachers feel the subject of prudent antimicrobial prescribing should be prioritized. However, teachers cited time restriction as the most important obstacle.3 Electronic/internet-based learning (e-learning) offers an interesting potential solution to this problem because, after creation, large groups of students can participate with a relatively small investment of time and cost.4,5 E-learning can be equally effective as an alternative education method in improving patient-related outcomes and influencing healthcare professionals.6 E-learning has shown positive effects on drug prescribing, but more evidence regarding behaviour change in practice and long-term retention (>12 weeks) is needed.4,7–9 Educational interventions on antimicrobial prescribing have shown effectiveness,10–14 but effects could not always be isolated from other intervention effects. The question remains whether a short period of e-learning can improve antimicrobial prescribing skills and behaviour in undergraduates in the long term. We developed a problem-based, interactive e-learning module on antimicrobial prescribing for fourth year medical students, conforming to scientific recommendations.2,15,16 We tested the e-learning’s long-term (after 6 months) effectiveness in improving student competence in performing a therapeutic infectious disease consultation in a simulated patient situation. Methods Design Prospective controlled intervention study of long-term effects, combined with randomized controlled intervention study of short-term effects (knowledge only). Population Medical students at the VU University Medical Centre Amsterdam, The Netherlands. Participant selection and study groups The e-learning module was introduced in the fourth year (out of six) of the medical curriculum between September 2011 and August 2012. Students were informed about the e-learning programme during an education lecture and asked to complete an antimicrobial knowledge pre-test. Afterwards, students were randomized to either direct e-learning access for 6 weeks or no e-learning (control group 1). E-learning group students received an e-mail link to the e-learning and up to three reminder e-mails in the case of <80% e-learning completion. E-learning access ended shortly before the antimicrobial knowledge post-test. To allow each student equal access to education, students in control group 1 were given access to the e-learning after the post-test. Owing to the potential risk of exposure to the e-learning (i.e. contamination), control group 1 was excluded from the long-term analysis. We created another control group (control group 2) for the long-term effect analysis of all students not included in the intervention group or control group 1 who had started their fourth year of the curriculum between September 2009 and September 2012. Figure S1 (available as Supplementary data at JAC Online) shows the full inclusion process for each study group. The study was approved by the national educational ethical review board (NVMO-ERB). E-learning module The e-learning module was built into the Dutch e-learning portal MedischOnderwijs.nl (in Dutch, to access: click https://www.medischonderwijs.nl? LESSONID=1693, register for free and click link again), comprised eight clinical cases and was based on the WHO’s guide to good prescribing,17 similar to the paradigm of the pharmacotherapy education in the curriculum. The e-learning included an evaluation survey (created in SurveyMonkey, http://www.surveymonkey.com). Measurements The antimicrobial knowledge tests comprised 57 multiple-choice questions each, validated by several experts. The objective structured clinical examination (OSCE) aimed to simulate prescribing behaviour in clinical practice and was set up concordant with recommendations from literature including use of a patient actor;18 also see the Supplementary methods and previous literature.19 The final product of the exam was a written prescription for an infectious disease case. Students were scored on overall performance based on a standardized score system including subscores for drug choice and knowledge. Examiners were blinded to group allocation. Outcomes Primary outcomes were based on evaluation of long-term e-learning effects and comprised overall drug choice and knowledge OSCE scores (all grades 1–10, higher scores indicating a better performance) and OSCE pass percentage (overall score >5.5). Secondary outcomes were the short-term effect of e-learning on knowledge by comparing scores on the second antimicrobial knowledge test while controlling for scores on the first test; and students’ evaluation of the e-learning. Statistical analysis We used the intention-to-treat principle to define intervention status. We determined the effect of the intervention by comparing outcomes between the intervention group and control group 1 for short-term effects and control group 2 for long-term effects. Univariate linear or logistic regression was used for all comparisons. We used propensity scores to control for confounding effects of students’ prior level, case differences and use of different examiners to allow valid comparisons. We also performed an as-treated analysis. We considered P < 0.05 significant. See the Supplementary methods for more details on the medical curriculum, outcome measures and statistics. Results Inclusion We included 56 students in control group 1, 68 students in the e-learning group for the short-term comparison, 285 students in control group 2 and 71 students in the e-learning group for the long-term comparison. Details of inclusion are shown in Figure S1. Baseline characteristics of study groups are shown in Table 1. Table 1. Baseline characteristics of study groups Short-term effects analysis e-learning group control group 1 total Number 68 56 124 Female, n (%) 49 (72) 43 (77) 92 (74) Age (years), average (range) 23.5 (20–38) 23.4 (21–48) 23.5 (20–48) Average score on antimicrobial test 1 (range) 4.0 (1.7–7.4) 4.0 (1.7–6.0) 4.0 (1.7–7.4) Average score on pharmacotherapy exam in prior year (range) 6.7 (3.9–9.1) 6.4 (2.5–8.8) 6.6 (2.5–9.1) E-learning, n (%)  ever opened 56 (82) — —  up to 25% completed 11 (16) — —  25%–75% completed 17 (25) — —  75%–100% completed 28 (41) — — Long-term effects analysis e-learning group control group 2 total Number 71 285 356 Female, n (%) 52 (73) 200 (70) 252 (71) Age (years), average (range) 23.5 (20–38) 23.1 (20–41) 23.2 (20–41) Average score on pharmacotherapy exam in prior year (range) 6.7 (2.8–9.1) 6.8 (2.5–9.6) 6.8 (2.5–9.6) E-learning, n (%)  ever opened 58 (82) — —  up to 25% completed 11 (15) — —  25%–75% completed 18 (25) — —  75%–100% completed 29 (41) — — Short-term effects analysis e-learning group control group 1 total Number 68 56 124 Female, n (%) 49 (72) 43 (77) 92 (74) Age (years), average (range) 23.5 (20–38) 23.4 (21–48) 23.5 (20–48) Average score on antimicrobial test 1 (range) 4.0 (1.7–7.4) 4.0 (1.7–6.0) 4.0 (1.7–7.4) Average score on pharmacotherapy exam in prior year (range) 6.7 (3.9–9.1) 6.4 (2.5–8.8) 6.6 (2.5–9.1) E-learning, n (%)  ever opened 56 (82) — —  up to 25% completed 11 (16) — —  25%–75% completed 17 (25) — —  75%–100% completed 28 (41) — — Long-term effects analysis e-learning group control group 2 total Number 71 285 356 Female, n (%) 52 (73) 200 (70) 252 (71) Age (years), average (range) 23.5 (20–38) 23.1 (20–41) 23.2 (20–41) Average score on pharmacotherapy exam in prior year (range) 6.7 (2.8–9.1) 6.8 (2.5–9.6) 6.8 (2.5–9.6) E-learning, n (%)  ever opened 58 (82) — —  up to 25% completed 11 (15) — —  25%–75% completed 18 (25) — —  75%–100% completed 29 (41) — — Table 1. Baseline characteristics of study groups Short-term effects analysis e-learning group control group 1 total Number 68 56 124 Female, n (%) 49 (72) 43 (77) 92 (74) Age (years), average (range) 23.5 (20–38) 23.4 (21–48) 23.5 (20–48) Average score on antimicrobial test 1 (range) 4.0 (1.7–7.4) 4.0 (1.7–6.0) 4.0 (1.7–7.4) Average score on pharmacotherapy exam in prior year (range) 6.7 (3.9–9.1) 6.4 (2.5–8.8) 6.6 (2.5–9.1) E-learning, n (%)  ever opened 56 (82) — —  up to 25% completed 11 (16) — —  25%–75% completed 17 (25) — —  75%–100% completed 28 (41) — — Long-term effects analysis e-learning group control group 2 total Number 71 285 356 Female, n (%) 52 (73) 200 (70) 252 (71) Age (years), average (range) 23.5 (20–38) 23.1 (20–41) 23.2 (20–41) Average score on pharmacotherapy exam in prior year (range) 6.7 (2.8–9.1) 6.8 (2.5–9.6) 6.8 (2.5–9.6) E-learning, n (%)  ever opened 58 (82) — —  up to 25% completed 11 (15) — —  25%–75% completed 18 (25) — —  75%–100% completed 29 (41) — — Short-term effects analysis e-learning group control group 1 total Number 68 56 124 Female, n (%) 49 (72) 43 (77) 92 (74) Age (years), average (range) 23.5 (20–38) 23.4 (21–48) 23.5 (20–48) Average score on antimicrobial test 1 (range) 4.0 (1.7–7.4) 4.0 (1.7–6.0) 4.0 (1.7–7.4) Average score on pharmacotherapy exam in prior year (range) 6.7 (3.9–9.1) 6.4 (2.5–8.8) 6.6 (2.5–9.1) E-learning, n (%)  ever opened 56 (82) — —  up to 25% completed 11 (16) — —  25%–75% completed 17 (25) — —  75%–100% completed 28 (41) — — Long-term effects analysis e-learning group control group 2 total Number 71 285 356 Female, n (%) 52 (73) 200 (70) 252 (71) Age (years), average (range) 23.5 (20–38) 23.1 (20–41) 23.2 (20–41) Average score on pharmacotherapy exam in prior year (range) 6.7 (2.8–9.1) 6.8 (2.5–9.6) 6.8 (2.5–9.6) E-learning, n (%)  ever opened 58 (82) — —  up to 25% completed 11 (15) — —  25%–75% completed 18 (25) — —  75%–100% completed 29 (41) — — E-learning effects Students in the e-learning group scored significantly higher on all continuous outcomes compared with control group students (Figure 1). OSCE pass percentage was also significantly higher for e-learning group students compared with control group students (97% versus 86%, n = 346, OR 5.90, 95% CI 1.74–20.01, P = 0.004). Effect sizes increased when using an as-treated approach (Figure S2). Figure 1. View largeDownload slide Continuous outcomes from students in the e-learning group compared with control group students. The possible grade range was 1–10, with higher scores indicating a better performance. Figure 1. View largeDownload slide Continuous outcomes from students in the e-learning group compared with control group students. The possible grade range was 1–10, with higher scores indicating a better performance. Evaluation survey Students rated the e-learning as instructive (average score from 1 to 10 was 7.4), 77% rated it as entirely relevant, but 55% rated it as too extensive. When questioned on their confidence in prescribing antimicrobial therapy in clinical practice prior to and subsequent to the e-learning, the percentage of students indicating insecurity or severe insecurity decreased from 74% to 37% (P = 0.002). Discussion This prospective controlled intervention study showed that access to e-learning for a limited time period significantly improved medical students’ long-term antimicrobial drug choice, antimicrobial knowledge and overall performance during an antimicrobial therapeutic consultation with a patient actor. In order to shape future antimicrobial prescriber behaviour, it is very important to identify resource-effective tools that can improve undergraduates’ prescribing competence, rather than at a later stage when physicians have already begun clinical practice.2 Our results suggest that e-learning may be just that. Although the main results on the continuous outcomes were relatively small (0.31–0.51 difference on a 1–10 scale; Figure 1), results on the pass percentage (11% difference) suggest that the intervention made a difference where it matters: among students whose prescribing competence balanced around the fail/pass level. The e-learning was rated positively by students and increased their self-rated confidence in prescribing. Our study is unique in that it estimates intervention impact by assessing students’ behaviour in a situation that simulates prescribing in clinical practice, using an approach recommended in literature.18 It is one of very few studies to evaluate the long-term impact (>12 weeks) of a temporary educational intervention on antimicrobial prescribing.11 The design of the study and the availability of prior student grades allowed us to create optimal equal comparisons between intervention and control groups. The inclusion of the short-term effect analysis made it possible to assess direct impact on knowledge and a potential fade of knowledge retention over time. The results of the intention-to-treat analyses show that intervention impact was significant when averaged for the whole group, although a considerable percentage of students either did not access the e-learning at all or only partially. This reflects intervention impact in practice, as participation in non-compulsory education is seldom expected to be perfect. The as-treated results suggest the ‘true effect’ is higher, as would be expected. The e-learning used comprised certain factors that may have enhanced its effectiveness, which are important to mention for future replication attempts.5 They include its problem-based architecture;2,15 the inclusion of interactive elements, exercises and feedback;16 and the use of the WHO six step plan to good prescribing, which is a method known to improve therapeutic competence of medical students.17,20 Moreover, the e-learning was implemented in the curriculum with pharmacotherapy education using the same paradigm.20 Although the curriculum was identical for the control group students, the interaction between e-learning and the curriculum may have supported long-term retention of learning effects. We need to address some limitations. Assessment of students’ skills is subjective and can lead to inconsistent results, which we aimed to diminish by using a structured assessment approach and adjustment for examiner effects. The results of the antimicrobial knowledge post-test were low across groups, suggesting the test may have been too difficult. This may have caused an imperfect measurement of student knowledge level. Finally, because our study included students who had to study for an additional 2 years after the OSCE, it is unclear how much of the e-learning effect will be present when they start prescribing in practice. Repeating the e-learning, for instance in postgraduate training, may be important. We have shown that e-learning access during a limited time period can significantly improve medical students’ long-term antimicrobial drug choice, antimicrobial knowledge and overall performance of an antimicrobial therapeutic consultation in a situation simulating clinical practice. Acknowledgements A selection of these results was presented at IDWeek 2016, New Orleans, LA, USA (Abstract 59757).  We wish to thank all experts who contributed to the development of the e-learning and the antimicrobial knowledge tests. We wish to thank Th. P. G. M. de Vries for his earlier contributions to the project. Funding This work was only supported by internal funding. Transparency declarations None to declare. Author contributions J. J. S. performed the statistical analysis and participant inclusion. All authors (J. J. S., M. G. C., T. S., M. H. H. K., J. T. and M. A. v. A.) were involved in data interpretation and writing the article. Supplementary data Figures S1 and S2 and Supplementary methods are available as Supplementary data at JAC Online. References 1 Dellit TH , Owens RC , McGowan JE Jr et al. Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America guidelines for developing an institutional program to enhance antimicrobial stewardship . Clin Infect Dis 2007 ; 44 : 159 – 77 . Google Scholar CrossRef Search ADS PubMed 2 Pulcini C , Gyssens IC. How to educate prescribers in antimicrobial stewardship practices . Virulence 2013 ; 4 : 192 – 202 . Google Scholar CrossRef Search ADS PubMed 3 Dyar OJ , Pulcini C , Howard P et al. European medical students: a first multicentre study of knowledge, attitudes and perceptions of antibiotic prescribing and antibiotic resistance . J Antimicrob Chemother 2014 ; 69 : 842 – 6 . Google Scholar CrossRef Search ADS PubMed 4 Rocha-Pereira N , Lafferty N , Nathwani D. Educating healthcare professionals in antimicrobial stewardship: can online-learning solutions help? J Antimicrob Chemother 2015 ; 70 : 3175 – 7 . Google Scholar CrossRef Search ADS PubMed 5 Gordon M , Chandratilake M , Baker P. Low fidelity, high quality: a model for e-learning . Clin Teach 2013 ; 10 : 258 – 63 . Google Scholar CrossRef Search ADS PubMed 6 Cook DA , Levinson AJ , Garside S et al. Internet-based learning in the health professions: a meta-analysis . JAMA 2008 ; 300 : 1181 – 96 . Google Scholar CrossRef Search ADS PubMed 7 Maxwell S , Mucklow J. e-Learning initiatives to support prescribing . Br J Clin Pharmacol 2012 ; 74 : 621 – 31 . Google Scholar CrossRef Search ADS PubMed 8 Keijsers CJPW , van Doorn ABD , van Kalles A et al. Structured pharmaceutical analysis of the Systematic Tool to Reduce Inappropriate Prescribing is an effective method for final-year medical students to improve polypharmacy skills: a randomized controlled trial . J Am Geriatr Soc 2014 ; 62 : 1353 – 9 . Google Scholar CrossRef Search ADS PubMed 9 Gordon M , Chandratilake M , Baker P. Improved junior paediatric prescribing skills after a short e-learning intervention: a randomised controlled trial . Arch Dis Child 2011 ; 96 : 1191 – 4 . Google Scholar CrossRef Search ADS PubMed 10 Davey P , Marwick CA , Scott CL et al. Interventions to improve antibiotic prescribing practices for hospital inpatients . Cochrane Database Syst Rev 2017 ; issue 2 : CD003543 . Google Scholar PubMed 11 Pérez-Cuevas R , Guiscafré H , Muñoz O et al. Improving physician prescribing patterns to treat rhinopharyngitis. Intervention strategies in two health systems of Mexico . Soc Sci Med 1996 ; 42 : 1185 – 94 . Google Scholar CrossRef Search ADS PubMed 12 Irfan N , Brooks A , Mithoowani S et al. A controlled quasi-experimental study of an educational intervention to reduce the unnecessary use of antimicrobials for asymptomatic bacteriuria . PLoS One 2015 ; 10 : e0132071. Google Scholar CrossRef Search ADS PubMed 13 Razon Y , Ashkenazi S , Cohen A et al. Effect of educational intervention on antibiotic prescription practices for upper respiratory infections in children: a multicentre study . J Antimicrob Chemother 2005 ; 56 : 937 – 40 . Google Scholar CrossRef Search ADS PubMed 14 Zamin MT , Pitre MM , Conly JM. Development of an intravenous-to-oral route conversion program for antimicrobial therapy at a Canadian tertiary care health facility . Ann Pharmacother 1997 ; 31 : 564 – 70 . Google Scholar CrossRef Search ADS PubMed 15 Gould IM , van der Meer JWM. Antibiotic Policies . New York, USA : Springer Science & Business Media , 2006 . 16 Cook DA , Levinson AJ , Garside S et al. Instructional design variations in internet-based learning for health professions education: a systematic review and meta-analysis . Acad Med 2010 ; 85 : 909 – 22 . Google Scholar CrossRef Search ADS PubMed 17 de Vries TPGM , Henning RH , Hogerzeil HV et al. Guide to Good Prescribing . Geneva, Switzerland : WHO , 1994 . http://apps.who.int/medicinedocs/pdf/whozip23e/whozip23e.pdf. 18 Maxwell SRJ. How should teaching of undergraduates in clinical pharmacology and therapeutics be delivered and assessed? Br J Clin Pharmacol 2012 ; 73 : 893 – 9 . Google Scholar CrossRef Search ADS PubMed 19 Brinkman DJ , Tichelaar J , van Agtmael MA et al. Self-reported confidence in prescribing skills correlates poorly with assessed competence in fourth-year medical students . J Clin Pharmacol 2015 ; 55 : 825 – 30 . Google Scholar CrossRef Search ADS PubMed 20 Richir MC , Tichelaar J , Stam F et al. A context-learning pharmacotherapy program for preclinical medical students leads to more rational drug prescribing during their clinical clerkship in internal medicine . Clin Pharmacol Ther 2008 ; 84 : 513 – 6 . Google Scholar CrossRef Search ADS PubMed © The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Antimicrobial Chemotherapy. All rights reserved. For permissions, please email: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)
TI - Improving antibiotic prescribing skills in medical students: the effect of e-learning after 6 months
JF - Journal of Antimicrobial Chemotherapy
DO - 10.1093/jac/dky163
DA - 2018-05-09
UR - https://www.deepdyve.com/lp/oxford-university-press/improving-antibiotic-prescribing-skills-in-medical-students-the-effect-dFD3S8VPJp
SP - 1
EP - 2246
VL - Advance Article
IS - 8
DP - DeepDyve
ER -