Evaluation of virtual patient cases for teaching diagnostic and management skills in internal medicine: a mixed methods study

Evaluation of virtual patient cases for teaching diagnostic and management skills in internal... Objective: The virtual patient ( VP) is a computer program that simulates real-life clinical scenarios and allows learners to make diagnostic and therapeutic decisions in a safe environment. Although many VP cases are available, few focus on junior trainees as their target audience. In addition, there is wide variability in trainees’ clinical rotation experi- ences, based on local practice and referral patterns, duty hour restrictions, and competing educational requirements. In order to standardize clinical exposure and improve trainees’ knowledge and perceived preparedness to manage core internal medicine cases, we developed a pool of VP cases to simulate common internal medicine presentations. We used quantitative and qualitative analyses to evaluate the effectiveness of one of our VP cases among medical trainees at University of Toronto. We also evaluated the role of VP cases in integrated teaching of non-medical expert competencies. Results: Despite modest effects on knowledge acquisition, a majority of participants enjoyed using VP cases as a resource to help them prepare for and reinforce clinical experiences. Cognitive interactivity and repetitive practice were particularly appreciated by study participants. Trainees perceived VP cases as a useful resource as their learning can be customized to their actions within the case, resulting in unique learning trajectories. Keywords: Virtual patients, Medical education, CanMEDS, Medical curriculum, Internal medicine Introduction competencies with significant impact on health-care The rapid evolution of medical knowledge, decreased delivery and patient satisfaction [5]. In the 1990s, the time for medical training, and ethical concerns about Royal College of Physicians and Surgeons of Canada patients as educational subjects have increased the com- developed the “Canadian Medical Education Directives plexity of medical decision making and medical training for Specialists” (CanMEDS) framework [reviewed in 6]. [1, 2]. The virtual patient (VP) is a computer program Training programs have implemented curricula to inte- that simulates real-life clinical scenarios and allows learn- grate the framework [6, 7]; however, there remains a ers to emulate the roles of health care providers to make paucity of literature on effective means of integrating the clinical decisions (reviewed in [3, 4]). While VP cases are CanMEDS framework in medical education. available widely online, few focus on medical students To standardize clinical exposure and improve train- and junior residents as their target audience. ees’ knowledge and perceived preparedness to manage In addition to the knowledge and technical exper- core internal medicine cases, we developed a pool of VP tise medical trainees must acquire, there are intrinsic cases to simulate internal medicine presentations. We used quantitative and qualitative analyses to evaluate the effectiveness of one of our VP cases. We also evaluated *Correspondence: samira.jeimy@lhsc.on.ca the role of VP cases as a tool for the integrated teaching Division of Clinical Immunology and Allergy, Department of General of CanMEDS competencies. Internal Medicine, Western University, B3-110, 268 Grosvenor Street, London, ON N6A 1M1, Canada Full list of author information is available at the end of the article © The Author(s) 2018. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creat iveco mmons .org/licen ses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creat iveco mmons .org/ publi cdoma in/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Jeimy et al. BMC Res Notes (2018) 11:357 Page 2 of 7 analyzed the focus group audio recordings using the Main text “framework” technique (described in [8]). Methods Module design We selected a VP case from a pool developed by physi- Results cians at the University of Toronto. Each module begins Quantitative analysis with defined learning objectives, followed by the case A total of 52 participants completed the study. Baseline and guided questions. Drop-down menus provide sug- characteristics were similar between groups (Table  1). gested responses to questions, and discussion points A majority had used online learning modules, but highlight concepts pertinent to evidence-based medi- most of these were non-interactive (Table  1). Partici- cine and key psychosocial factors. For this study, we pants differed in their perceived confidence in diagnos - used a case on diagnosis and management of upper gas- ing UGIB, with lower baseline ability to diagnose and trointestinal bleed (UGIB). manage UGIB in trainees who completed the VP case (Table 1). Table  2 compares average objective knowledge acqui- Module evaluation sition and clinical reasoning scores from participants’ We invited University of Toronto trainees to partici- post-module tests. Trainees performed similarly in each pate in VP case evaluation. Trainees completed ques- test question, including those pertaining to non-medi- tionnaires about demographic items and rated their cal-expert CanMEDS competencies (Table  2). Change perceived confidence in diagnostic and management in level of confidence for clinical hand-over trended abilities (Additional file  1: Additional materials— towards being higher in the VP-case arm (Mann–Whit- Questionnaires). Items were measured on a 5-point ney P = 0.051, Table 2). Overall, there were no significant scale ranging from 1 (“poor” confidence) to 5 (“excel - differences in participants’ perceived confidence (Addi - lent” confidence). Next, we randomized trainees to tional file 2: Tables S1 and Additional file 3: Table S2). complete a VP case (intervention arm) or PowerPoint We asked participants for feedback on the VP case. presentation (control arm). After module completion, Although trainees thought there was similar learning participants again completed the confidence question - value in the VP case and PowerPoint presentation, they naire, and a 10-item multiple-choice test assessing preferred the VP case as a learning resource (Additional knowledge acquisition (Additional file  1: Additional file 4: Tables S3 and Additional file 5: Table S4). materials—Questionnaires). We invited trainees who completed the post-test to participate in audiotaped focus groups to provide Qualitative analysis open-ended feedback, especially related to their expe- To further evaluate how medical students and residents riences in learning the integrated non-medical expert used the VP cases in their learning, especially pertaining CanMEDS roles. We conducted two focus groups (5 to integration of non-medical-expert CanMEDS com- participants each) with medical students and residents, petencies, we organized two trainee focus groups. The using a semi-structured interview method. Each meet- baseline characteristics of these participants are shown ing was between 60 and 120 min in duration. Meetings in Additional file  6: Table S5. Seven categories of investi- were audiotaped and transcribed. gation were highlighted regarding VP cases as a learning The methods were approved by the institutional resource and for integration of CanMEDS competencies. research ethics board at the University of Toronto. We divided each category into subcategories; each com- Written consent was obtained from all participants. ment was allocated to one of these subcategories. We repeated charting for focus groups using the same sub- Data analysis categories. The interview technique was iterative after We analyzed responses to the multiple-choice ques- the first focus group. We (JW and SJ) compared individ - tions with the Wilcoxon signed-rank test using Excel ual analysis of the focus group comments, and found our software. Free text responses were analyzed using analyses in agreement. The categories and subcategories grounded theory to identify common themes. Two of the framework technique are presented in Table 3 and authors (SJ and JW) independently transcribed and below. Jeimy et al. BMC Res Notes (2018) 11:357 Page 3 of 7 Table 1 Participant characteristics (% of  participants Table 1 (continued) in each intervention group) VP Case PowerPoint Chi square (n = 23) (n = 29) P value VP Case PowerPoint Chi square (n = 23) (n = 29) P value Unremarkable 43 31 Poor 22 17 Level of training 0.799 1–2 years medical student 13 17 VP virtual patient, UGIB upper gastrointestinal bleed, OGD oesophago-gastro- duodenoscopy 3–4 years medical student 61 52 Resident 26 31 Time from last IM rotation 0.965 Category 1. Trainees are looking for practical resources No previous IM rotation 26 24 beyond didactic lectures > 6 months 13 10 Focus group participants emphasized limited time to 3–6 months 13 17 learn a vast amount of knowledge. The transition points < 3 months 26 21 from pre-clerkship to clerkship, and from senior medical Currently in IM rotation 22 28 student to intern, was felt to be especially challenging. Initial objectives N/A I think one of the hardest parts when you are Review knowledge 87 90 beginning is … we get inundated with so much Acquire new medical expert 65 52 knowledge information. We don’t know what is important and Improve non-medical expert 17 10 what is not. competency Application of knowledge 52 66 Recruited for research 4 0 Previous exposure to virtual patients 0.313 Category 2. Learning needs differ based on level of training None 87 76 Pre-clinical students were looking to gain experience Any virtual patient cases 13 24 with a practical approach to a clinical presentation. Self evaluation of ability to: I’m focusing less about the UGIB content and more Diagnose UGIB 0.064 about the experience – I think I took a lot out of it Excellent 0 3 that way Very good 43 28 Satisfactory 35 59 Most junior medical students wanted repeated prac- Unremarkable 4 10 tice with ward skills. Senior medical students con- Poor 17 0 firmed that they felt inadequately prepared, which Manage UGIB 0.394 detracted from their learning experience. Excellent 0 3 Things like handover, writing admission orders, Very good 22 14 Satisfactory 48 66 writing a prescription, etc. - we didn’t get taught Unremarkable 9 10 that until the week before clerkship started. Poor 22 7 Senior medical students were looking for efficient Handover 0.340 ways to refresh their knowledge, especially in prepara- Excellent 4 3 tion for their licensing exams. They emphasized that an Very good 26 14 interactive platform would be more effective than pas - Satisfactory 48 69 sive review of lectures or of the scientific literature. Unremarkable 13 14 The consensus was that learning resources which Poor 9 0 provide concise, practical, evidence-based informa- Write admission orders 0.284 tion would be useful to help build confidence in diag - Excellent 4 3 nostic and management skills. Trainees emphasized Good 26 10 the importance of a simulation setting where they can Satisfactory 43 55 safely practice skills without consequences on patient Unremarkable 13 28 care. Poor 13 3 OGD consent 0.518 It is nice to go through early in clerkship, to have a Excellent 0 0 place to safely practice things without judgement or Very good 13 10 killing a patient. Satisfactory 22 41 Jeimy et al. BMC Res Notes (2018) 11:357 Page 4 of 7 Table 2 Comparing objective assessment scores (median [IQR1, IQR3]) Question topics VP case (n = 23) PowerPoint (n = 29) Mann–Whitney U test P-value Sign over 1.00 (1.00–1.00) 1.00 (1.00–1.00) 0.979 Hypovolemia signs 1.00 (0.00–1.00) 0.00 (0.00–1.00) 0.284 Most important step in management 0.00 (0.00–0.50) 0.00 (0.00–0.00) 0.650 Risk scores 1.00 (1.00–1.00) 1.00 (1.00–1.00) 0.832 Presentation of UGIB 1.00 (0.80–1.00) 0.80 (0.60–1.00) 0.274 Complications of OGD 0.75 (0.75–1.00) 0.75 (0.50–1.00) 0.210 Bad exam manoeuvre 1.00 (0.00–1.00) 1.00 (0.00–1.00) 0.335 High risk OGD lesion 0.75 (0.50–1.00) 0.75 (0.50–1.00) 0.909 Post OGD monitoring 1.00 (0.00–1.00) 0.00 (0.00–1.00) 0.073 VP virtual patient, UGIB upper gastrointestinal bleed, OGD oesophago-gastro-duodenoscopy Table 3 Focus group categories Categories Subcategories Residents want practical resources beyond traditional curriculum Want concise, evidence-based, clinically relevant information Place to practice skills without consequences Medical students at different levels have different learning needs Preclinical students are focused on tips/skills Preclinical students want to practice experience of real world before clerkship Clerks are focused on knowledge/medical expert content Clerks want to practice application of knowledge Difficult to meet needs with any one type of learning resource Appreciated elements of IMCE cases High quality, comprehensive Realistic Practical delivery of clinically relevant details Provides an approach Evidence-based Interactive Optional curriculum resource Suggestions for improvement Cases are too long, with too many details e.g. scoring systems Link to multimedia (videos, images, Apps) Include extra information like scoring systems as optional links Increase interactivity General challenges in the current use of CanMEDS in medical education The way CanMEDS breaks down the concept of the physician is reductionist, not organic Portfolio—allows debriefing on challenging cases, but rigid format CanMEDS is useful for educators to plan curriculum but may be inherently challenging to teach VP cases and CanMEDS VP cases may be a useful resource to integrate CanMEDS roles Simulations cannot replace real world experience of patient care Some skills are still better learned via practice and experience Category 3. VP cases are a useful adjunct to didactic lectures Toronto Notes is … comprehensive but a book of In comparison to commonly-used resources, medical lists with no emphasis on what’s common, what you students appreciated that the VP case simulated a real should prioritize. This realistic case scenario which world clinical scenario. takes you through the steps in practical terms is more useful. Jeimy et al. BMC Res Notes (2018) 11:357 Page 5 of 7 Residents appreciated the teaching of practical, clini- Discussion cally-relevant details. Our goal in creating VP cases was to facilitate transi- tion between the role of a senior medical student and a I like the specifics - like doses and timelines, like first year internal medicine resident. This was based on 72 hours, how many milligram. At a resident level consensus among colleagues and studies reporting that that’s what we need to know. 41–60% of medical graduates feel clinically unprepared Participants also liked the integration of evidence- after medical school graduation [9–11]. Interactive VP based medicine, and appreciated the user-friendly, inter- cases allows medical educators to facilitate learning in active aspect of the case. an environment that does not compromise patient safety [12]. We hoped that our VP cases could complement the And the format you used with the dropdown menus medical curricula to help trainees become comfortable – they gave you a chance to think about the question with assessing and managing common, key presentations and then the answer was there. in a protected environment. Our results indicated that VP cases did not significantly affect knowledge acquisition, for both medical expert and Category 4. Suggestions for improvement non-medical expert CanMEDS topics (Table  2). This is Trainees felt the case was lengthy and contained extra- consistent with a meta-analysis of 201 studies summa- neous detail. Participants also wanted a greater level of rizing the effect of internet-based instruction for medi - interactivity. cal trainees [13]. Despite its modest effects on knowledge acquisition, a majority of participants enjoyed using VP Category 5. There are challenges to current approaches cases as a resource to help them prepare for, and rein- to CanMEDS training force clinical experiences. Trainees’ preference for using Participants felt that the way CanMEDS breaks down the VP cases, especially over traditional curriculum adjuncts, concept of the physician is reductionist, and not realistic. is important, as learner engagement can significantly CanMEDS is trying to make an abstract thing con- improve effectiveness of technology-enhanced simulation crete and it does not make sense. If you try to focus [14]. Other features of simulation-based training shown on communicator role in our job as a physician, it is to be effective in medical education, including cognitive not doing it justice. We communicate all the time, it interactivity and repetitive practice [13], were aspects of is hard to take it out of context and isolate it. our VP case appreciated by study participants. It is possi- ble that trainees perceive VP cases as a useful resource as their learning can be customized to their actions within Category 6. VPs may represent a useful tool for integrating the case, resulting in unique learning trajectories. For CanMEDS example, in our study junior trainees focused on learning The consensus was that VP cases may be a useful an approach to the consultation process, whereas senior resource to integrate CanMEDS roles in medical educa- trainees reviewed their medical knowledge. In addition, tion. Trainees appreciated that they did not realize they junior trainees concentrated on non-medical expert Can- were learning CanMEDS competencies throughout the MEDS competencies, such as writing admission orders, VP case. whereas senior trainees enjoyed learning about evi- It was a surprise. For example, writing admission dence-based medicine. Based on current adult learning orders. Those are really useful for clerkship. Nice way theories, such personalized and interactive instruction to integrate it without being explicit. methods may be more powerful and efficient than didac - tic education [15]. They also liked that multiple competencies were cov - Although trainees agreed that non-medical expert ered with one concept. CanMEDS roles are important, they consistently I like that about [the] cases. Like handover expressed dissatisfaction with existing CanMEDS cur- includes…communication, collaboration… ricula, finding the approaches reductionist and artificial. Trainees appreciated incorporation of CanMEDS topics in the VP case, and especially that multiple CanMEDS Category 7. Simulations cannot replace real world experience competencies were introduced without disrupting the of patient care flow of the case. VP cases may provide an exciting new Most trainees felt that, although VP cases provided a arena where CanMEDS competencies can be introduced useful adjunct, many of the CanMEDS competencies are or reinforced. best achieved through real world experience. Jeimy et al. BMC Res Notes (2018) 11:357 Page 6 of 7 Authors’ contributions There are several advantages to integrating VP cases SJ and JW collected, analyzed and interpreted the data. SJ prepared the initial in medical education, including cost benefits, cases that draft of the manuscript. JW and LR were major contributors in developing the closely match real-life situations, the ability to create col- research design, analyzing the data, and critically evaluating the manuscript, and agree to be accountable for all aspects of the work. All authors read and lections of similar cases, seamless integration of Can- approved the final manuscript MEDS competency training, and the ability to create VP cases with which trainees should ideally gain compe- Author details Division of Clinical Immunology and Allergy, Department of General Internal tence. Future work will concentrate case enhancements Medicine, Western University, B3-110, 268 Grosvenor Street, London, ON N6A based on feedback, and cases that can provide real-time 1M1, Canada. Department of General Internal Medicine, University Health feedback or introduce different challenges based on Network - Toronto General Hospital, University of Toronto, EN 14-218, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada. training levels. We hope to create a larger pool of cases to allow for standardization in trainees’ exposure to com- Acknowledgements mon and atypical internal medicine presentations. The authors are grateful to Dr. Michael Li for his contributions to the creation of the virtual patient cases. Competing interests Limitations The authors declare that they have no competing interests. Our analysis is limited by a small sample size and selec- Availability of data and materials tive participation. Another limitation is the use of The datasets generated and analyzed during the current study are available self-assessment to evaluate changes in knowledge and from the corresponding author on reasonable request. confidence in managing UGIB, although analysis of Consent for publication objective knowledge scores corroborated subjective Not applicable. reports. Lastly, our study was limited to trainees at Uni- versity of Toronto, and used one VP case to extrapolate Ethics approval and consent to participate The methods outlined were approved by the institutional research ethics data. It would be interesting to evaluate whether our data board at the University of Toronto. Written consent was obtained from all is reproducible for different VP cases, and at other medi - study participants. cal training programs. Funding None to declare. Additional files Publisher’s Note Additional file 1: Additional Questionaries. Assessment of participant Springer Nature remains neutral with regard to jurisdictional claims in pub- characteristics, self-evaluation, knowledge assessment, and case-based lished maps and institutional affiliations. feedback. Questionnaires developed for the study, to assess participant demographics, perceived confidence in diagnostic and management Received: 13 April 2018 Accepted: 1 June 2018 abilities, knowledge, and feedback on the virtual patient case. Additional file 2: Table S1. Change in Level of Confidence after Inter - vention (Median Change in Likert Scale Rating [IQR1, IQR3]). Trainees’ perceived self-confidence in diagnostic and management abilities, measured on a 5-point rating scale ranging from 1 (“poor” confidence) to References 5 (“excellent” confidence). Median change before and after completing 1. Reed DA, Levine RB, Miller RG, Ashar BH, Bass EB, Rice TN, Cofrancesco J the virtual patient case are reported. Jr. Eec ff t of residency duty-hour limits: views of key clinical faculty. Arch Intern Med. 2007;167:1487–92. Additional file 3: Table S2. Participant Characteristics for Participants 2. Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: who Completed Pre-test Only vs. Pre- and Post- Test (% of Participants in an ethical imperative. Acad Med. 2003;78:783–8. Each Intervention Group). Demographic characteristics and perceived 3. McGee JB, Neill J, Goldman L, Casey E. Using multimedia virtual patients confidence of participants who completed the pre-test only, with those to enhance the clinical curriculum for medical students. Stud Health who completed both the pre- and post- tests, to confirm that these Technol Inform. 1998;52(Pt 2):732–5. groups were not significantly different. 4. Simo A, Cavazza M, Kijima R. Virtual patients in clinical medicine. Stud Additional file 4: Table S3. Participant Case Evaluation (% of Participants Health Technol Inform. 2004;98:353–9. in Each Intervention Group). Trainees’ evaluation of the virtual patient case, 5. Vincent C, Young M, Phillips A. Why do people sue doctors? A study of based on questionnaires developed for the study. patients and relatives taking legal action. Lancet. 1994;343:1609–13. Additional file 5: Table S4. Themes in Free-Text Feedback. Trainees’ free- 6. Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes- text evaluation of the virtual patient case. based framework of physician competencies. Med Teach. 2007;29:642–7. 7. Chou S, Cole G, McLaughlin K, Lockyer J. CanMEDS evaluation in Cana- Additional file 6: Table S5. Focus group participant characteristics. dian postgraduate training programmes: tools used and programme Demographic features of trainees who participated in focus groups. director satisfaction. Med Educ. 2008;42:879–86. 8. Ritchie J, Spencer L. Analyzing Qualitative Data. In: Bryman A, Burgess RG, editors. Qualitative data analysis for applied policy research. London: Routledge; 1994. p. 173–94. Abbreviations 9. Cave J, Goldacre M, Lambert T, Woolf K, Jones A, Dacre J. Newly qualified VP: virtual patient; CanMEDS: Canadian Medical Education Directives for doctors’ views about whether their medical school had trained them Specialists; UGIB: upper gastrointestinal bleed. well: questionnaire surveys. BMC Med Educ. 2007;7:38. Jeimy et al. BMC Res Notes (2018) 11:357 Page 7 of 7 10. Goldacre MJ, Davidson JM, Lambert TW. Doctors’ views of their first year 14. Issenberg SB, McGaghie WC, Petrusa ER, Lee GD, Scalese RJ. Features and of medical work and postgraduate training in the UK: questionnaire uses of high-fidelity medical simulations that lead to effective learning: a surveys. Med Educ. 2003;37:802–8. BEME systematic review. Med Teach. 2005;27:10–28. 11. Ochsmann EB, Zier U, Drexler H, Schmid K. Well prepared for work? 15. Davis D, Bordage G, Moores LK, Bennett N, Marinopoulos SS, Mazmanian Junior doctors’ self-assessment after medical education. BMC Med Educ. PE, Dorman T, McCrory D. The science of continuing medical education: 2011;11:99. terms, tools, and gaps: effectiveness of continuing medical education: 12. Cook DA, Hamstra SJ, Brydges R, Zendejas B, Szostek JH, Wang AT, Erwin American College of Chest Physicians Evidence-Based Educational PJ, Hatala R. Comparative effectiveness of instructional design features in Guidelines. Chest. 2009;135:8S–16S. simulation-based education: systematic review and meta-analysis. Med Teach. 2013;35:e867–98. 13. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA. 2008;300:1181–96. Ready to submit your research ? Choose BMC and benefit from: fast, convenient online submission thorough peer review by experienced researchers in your field rapid publication on acceptance support for research data, including large and complex data types • gold Open Access which fosters wider collaboration and increased citations maximum visibility for your research: over 100M website views per year At BMC, research is always in progress. Learn more biomedcentral.com/submissions http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png BMC Research Notes Springer Journals

Evaluation of virtual patient cases for teaching diagnostic and management skills in internal medicine: a mixed methods study

Free
7 pages
Loading next page...
 
/lp/springer_journal/evaluation-of-virtual-patient-cases-for-teaching-diagnostic-and-oPRXzapeLP
Publisher
BioMed Central
Copyright
Copyright © 2018 by The Author(s)
Subject
Biomedicine; Biomedicine, general; Medicine/Public Health, general; Life Sciences, general
eISSN
1756-0500
D.O.I.
10.1186/s13104-018-3463-x
Publisher site
See Article on Publisher Site

Abstract

Objective: The virtual patient ( VP) is a computer program that simulates real-life clinical scenarios and allows learners to make diagnostic and therapeutic decisions in a safe environment. Although many VP cases are available, few focus on junior trainees as their target audience. In addition, there is wide variability in trainees’ clinical rotation experi- ences, based on local practice and referral patterns, duty hour restrictions, and competing educational requirements. In order to standardize clinical exposure and improve trainees’ knowledge and perceived preparedness to manage core internal medicine cases, we developed a pool of VP cases to simulate common internal medicine presentations. We used quantitative and qualitative analyses to evaluate the effectiveness of one of our VP cases among medical trainees at University of Toronto. We also evaluated the role of VP cases in integrated teaching of non-medical expert competencies. Results: Despite modest effects on knowledge acquisition, a majority of participants enjoyed using VP cases as a resource to help them prepare for and reinforce clinical experiences. Cognitive interactivity and repetitive practice were particularly appreciated by study participants. Trainees perceived VP cases as a useful resource as their learning can be customized to their actions within the case, resulting in unique learning trajectories. Keywords: Virtual patients, Medical education, CanMEDS, Medical curriculum, Internal medicine Introduction competencies with significant impact on health-care The rapid evolution of medical knowledge, decreased delivery and patient satisfaction [5]. In the 1990s, the time for medical training, and ethical concerns about Royal College of Physicians and Surgeons of Canada patients as educational subjects have increased the com- developed the “Canadian Medical Education Directives plexity of medical decision making and medical training for Specialists” (CanMEDS) framework [reviewed in 6]. [1, 2]. The virtual patient (VP) is a computer program Training programs have implemented curricula to inte- that simulates real-life clinical scenarios and allows learn- grate the framework [6, 7]; however, there remains a ers to emulate the roles of health care providers to make paucity of literature on effective means of integrating the clinical decisions (reviewed in [3, 4]). While VP cases are CanMEDS framework in medical education. available widely online, few focus on medical students To standardize clinical exposure and improve train- and junior residents as their target audience. ees’ knowledge and perceived preparedness to manage In addition to the knowledge and technical exper- core internal medicine cases, we developed a pool of VP tise medical trainees must acquire, there are intrinsic cases to simulate internal medicine presentations. We used quantitative and qualitative analyses to evaluate the effectiveness of one of our VP cases. We also evaluated *Correspondence: samira.jeimy@lhsc.on.ca the role of VP cases as a tool for the integrated teaching Division of Clinical Immunology and Allergy, Department of General of CanMEDS competencies. Internal Medicine, Western University, B3-110, 268 Grosvenor Street, London, ON N6A 1M1, Canada Full list of author information is available at the end of the article © The Author(s) 2018. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creat iveco mmons .org/licen ses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creat iveco mmons .org/ publi cdoma in/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Jeimy et al. BMC Res Notes (2018) 11:357 Page 2 of 7 analyzed the focus group audio recordings using the Main text “framework” technique (described in [8]). Methods Module design We selected a VP case from a pool developed by physi- Results cians at the University of Toronto. Each module begins Quantitative analysis with defined learning objectives, followed by the case A total of 52 participants completed the study. Baseline and guided questions. Drop-down menus provide sug- characteristics were similar between groups (Table  1). gested responses to questions, and discussion points A majority had used online learning modules, but highlight concepts pertinent to evidence-based medi- most of these were non-interactive (Table  1). Partici- cine and key psychosocial factors. For this study, we pants differed in their perceived confidence in diagnos - used a case on diagnosis and management of upper gas- ing UGIB, with lower baseline ability to diagnose and trointestinal bleed (UGIB). manage UGIB in trainees who completed the VP case (Table 1). Table  2 compares average objective knowledge acqui- Module evaluation sition and clinical reasoning scores from participants’ We invited University of Toronto trainees to partici- post-module tests. Trainees performed similarly in each pate in VP case evaluation. Trainees completed ques- test question, including those pertaining to non-medi- tionnaires about demographic items and rated their cal-expert CanMEDS competencies (Table  2). Change perceived confidence in diagnostic and management in level of confidence for clinical hand-over trended abilities (Additional file  1: Additional materials— towards being higher in the VP-case arm (Mann–Whit- Questionnaires). Items were measured on a 5-point ney P = 0.051, Table 2). Overall, there were no significant scale ranging from 1 (“poor” confidence) to 5 (“excel - differences in participants’ perceived confidence (Addi - lent” confidence). Next, we randomized trainees to tional file 2: Tables S1 and Additional file 3: Table S2). complete a VP case (intervention arm) or PowerPoint We asked participants for feedback on the VP case. presentation (control arm). After module completion, Although trainees thought there was similar learning participants again completed the confidence question - value in the VP case and PowerPoint presentation, they naire, and a 10-item multiple-choice test assessing preferred the VP case as a learning resource (Additional knowledge acquisition (Additional file  1: Additional file 4: Tables S3 and Additional file 5: Table S4). materials—Questionnaires). We invited trainees who completed the post-test to participate in audiotaped focus groups to provide Qualitative analysis open-ended feedback, especially related to their expe- To further evaluate how medical students and residents riences in learning the integrated non-medical expert used the VP cases in their learning, especially pertaining CanMEDS roles. We conducted two focus groups (5 to integration of non-medical-expert CanMEDS com- participants each) with medical students and residents, petencies, we organized two trainee focus groups. The using a semi-structured interview method. Each meet- baseline characteristics of these participants are shown ing was between 60 and 120 min in duration. Meetings in Additional file  6: Table S5. Seven categories of investi- were audiotaped and transcribed. gation were highlighted regarding VP cases as a learning The methods were approved by the institutional resource and for integration of CanMEDS competencies. research ethics board at the University of Toronto. We divided each category into subcategories; each com- Written consent was obtained from all participants. ment was allocated to one of these subcategories. We repeated charting for focus groups using the same sub- Data analysis categories. The interview technique was iterative after We analyzed responses to the multiple-choice ques- the first focus group. We (JW and SJ) compared individ - tions with the Wilcoxon signed-rank test using Excel ual analysis of the focus group comments, and found our software. Free text responses were analyzed using analyses in agreement. The categories and subcategories grounded theory to identify common themes. Two of the framework technique are presented in Table 3 and authors (SJ and JW) independently transcribed and below. Jeimy et al. BMC Res Notes (2018) 11:357 Page 3 of 7 Table 1 Participant characteristics (% of  participants Table 1 (continued) in each intervention group) VP Case PowerPoint Chi square (n = 23) (n = 29) P value VP Case PowerPoint Chi square (n = 23) (n = 29) P value Unremarkable 43 31 Poor 22 17 Level of training 0.799 1–2 years medical student 13 17 VP virtual patient, UGIB upper gastrointestinal bleed, OGD oesophago-gastro- duodenoscopy 3–4 years medical student 61 52 Resident 26 31 Time from last IM rotation 0.965 Category 1. Trainees are looking for practical resources No previous IM rotation 26 24 beyond didactic lectures > 6 months 13 10 Focus group participants emphasized limited time to 3–6 months 13 17 learn a vast amount of knowledge. The transition points < 3 months 26 21 from pre-clerkship to clerkship, and from senior medical Currently in IM rotation 22 28 student to intern, was felt to be especially challenging. Initial objectives N/A I think one of the hardest parts when you are Review knowledge 87 90 beginning is … we get inundated with so much Acquire new medical expert 65 52 knowledge information. We don’t know what is important and Improve non-medical expert 17 10 what is not. competency Application of knowledge 52 66 Recruited for research 4 0 Previous exposure to virtual patients 0.313 Category 2. Learning needs differ based on level of training None 87 76 Pre-clinical students were looking to gain experience Any virtual patient cases 13 24 with a practical approach to a clinical presentation. Self evaluation of ability to: I’m focusing less about the UGIB content and more Diagnose UGIB 0.064 about the experience – I think I took a lot out of it Excellent 0 3 that way Very good 43 28 Satisfactory 35 59 Most junior medical students wanted repeated prac- Unremarkable 4 10 tice with ward skills. Senior medical students con- Poor 17 0 firmed that they felt inadequately prepared, which Manage UGIB 0.394 detracted from their learning experience. Excellent 0 3 Things like handover, writing admission orders, Very good 22 14 Satisfactory 48 66 writing a prescription, etc. - we didn’t get taught Unremarkable 9 10 that until the week before clerkship started. Poor 22 7 Senior medical students were looking for efficient Handover 0.340 ways to refresh their knowledge, especially in prepara- Excellent 4 3 tion for their licensing exams. They emphasized that an Very good 26 14 interactive platform would be more effective than pas - Satisfactory 48 69 sive review of lectures or of the scientific literature. Unremarkable 13 14 The consensus was that learning resources which Poor 9 0 provide concise, practical, evidence-based informa- Write admission orders 0.284 tion would be useful to help build confidence in diag - Excellent 4 3 nostic and management skills. Trainees emphasized Good 26 10 the importance of a simulation setting where they can Satisfactory 43 55 safely practice skills without consequences on patient Unremarkable 13 28 care. Poor 13 3 OGD consent 0.518 It is nice to go through early in clerkship, to have a Excellent 0 0 place to safely practice things without judgement or Very good 13 10 killing a patient. Satisfactory 22 41 Jeimy et al. BMC Res Notes (2018) 11:357 Page 4 of 7 Table 2 Comparing objective assessment scores (median [IQR1, IQR3]) Question topics VP case (n = 23) PowerPoint (n = 29) Mann–Whitney U test P-value Sign over 1.00 (1.00–1.00) 1.00 (1.00–1.00) 0.979 Hypovolemia signs 1.00 (0.00–1.00) 0.00 (0.00–1.00) 0.284 Most important step in management 0.00 (0.00–0.50) 0.00 (0.00–0.00) 0.650 Risk scores 1.00 (1.00–1.00) 1.00 (1.00–1.00) 0.832 Presentation of UGIB 1.00 (0.80–1.00) 0.80 (0.60–1.00) 0.274 Complications of OGD 0.75 (0.75–1.00) 0.75 (0.50–1.00) 0.210 Bad exam manoeuvre 1.00 (0.00–1.00) 1.00 (0.00–1.00) 0.335 High risk OGD lesion 0.75 (0.50–1.00) 0.75 (0.50–1.00) 0.909 Post OGD monitoring 1.00 (0.00–1.00) 0.00 (0.00–1.00) 0.073 VP virtual patient, UGIB upper gastrointestinal bleed, OGD oesophago-gastro-duodenoscopy Table 3 Focus group categories Categories Subcategories Residents want practical resources beyond traditional curriculum Want concise, evidence-based, clinically relevant information Place to practice skills without consequences Medical students at different levels have different learning needs Preclinical students are focused on tips/skills Preclinical students want to practice experience of real world before clerkship Clerks are focused on knowledge/medical expert content Clerks want to practice application of knowledge Difficult to meet needs with any one type of learning resource Appreciated elements of IMCE cases High quality, comprehensive Realistic Practical delivery of clinically relevant details Provides an approach Evidence-based Interactive Optional curriculum resource Suggestions for improvement Cases are too long, with too many details e.g. scoring systems Link to multimedia (videos, images, Apps) Include extra information like scoring systems as optional links Increase interactivity General challenges in the current use of CanMEDS in medical education The way CanMEDS breaks down the concept of the physician is reductionist, not organic Portfolio—allows debriefing on challenging cases, but rigid format CanMEDS is useful for educators to plan curriculum but may be inherently challenging to teach VP cases and CanMEDS VP cases may be a useful resource to integrate CanMEDS roles Simulations cannot replace real world experience of patient care Some skills are still better learned via practice and experience Category 3. VP cases are a useful adjunct to didactic lectures Toronto Notes is … comprehensive but a book of In comparison to commonly-used resources, medical lists with no emphasis on what’s common, what you students appreciated that the VP case simulated a real should prioritize. This realistic case scenario which world clinical scenario. takes you through the steps in practical terms is more useful. Jeimy et al. BMC Res Notes (2018) 11:357 Page 5 of 7 Residents appreciated the teaching of practical, clini- Discussion cally-relevant details. Our goal in creating VP cases was to facilitate transi- tion between the role of a senior medical student and a I like the specifics - like doses and timelines, like first year internal medicine resident. This was based on 72 hours, how many milligram. At a resident level consensus among colleagues and studies reporting that that’s what we need to know. 41–60% of medical graduates feel clinically unprepared Participants also liked the integration of evidence- after medical school graduation [9–11]. Interactive VP based medicine, and appreciated the user-friendly, inter- cases allows medical educators to facilitate learning in active aspect of the case. an environment that does not compromise patient safety [12]. We hoped that our VP cases could complement the And the format you used with the dropdown menus medical curricula to help trainees become comfortable – they gave you a chance to think about the question with assessing and managing common, key presentations and then the answer was there. in a protected environment. Our results indicated that VP cases did not significantly affect knowledge acquisition, for both medical expert and Category 4. Suggestions for improvement non-medical expert CanMEDS topics (Table  2). This is Trainees felt the case was lengthy and contained extra- consistent with a meta-analysis of 201 studies summa- neous detail. Participants also wanted a greater level of rizing the effect of internet-based instruction for medi - interactivity. cal trainees [13]. Despite its modest effects on knowledge acquisition, a majority of participants enjoyed using VP Category 5. There are challenges to current approaches cases as a resource to help them prepare for, and rein- to CanMEDS training force clinical experiences. Trainees’ preference for using Participants felt that the way CanMEDS breaks down the VP cases, especially over traditional curriculum adjuncts, concept of the physician is reductionist, and not realistic. is important, as learner engagement can significantly CanMEDS is trying to make an abstract thing con- improve effectiveness of technology-enhanced simulation crete and it does not make sense. If you try to focus [14]. Other features of simulation-based training shown on communicator role in our job as a physician, it is to be effective in medical education, including cognitive not doing it justice. We communicate all the time, it interactivity and repetitive practice [13], were aspects of is hard to take it out of context and isolate it. our VP case appreciated by study participants. It is possi- ble that trainees perceive VP cases as a useful resource as their learning can be customized to their actions within Category 6. VPs may represent a useful tool for integrating the case, resulting in unique learning trajectories. For CanMEDS example, in our study junior trainees focused on learning The consensus was that VP cases may be a useful an approach to the consultation process, whereas senior resource to integrate CanMEDS roles in medical educa- trainees reviewed their medical knowledge. In addition, tion. Trainees appreciated that they did not realize they junior trainees concentrated on non-medical expert Can- were learning CanMEDS competencies throughout the MEDS competencies, such as writing admission orders, VP case. whereas senior trainees enjoyed learning about evi- It was a surprise. For example, writing admission dence-based medicine. Based on current adult learning orders. Those are really useful for clerkship. Nice way theories, such personalized and interactive instruction to integrate it without being explicit. methods may be more powerful and efficient than didac - tic education [15]. They also liked that multiple competencies were cov - Although trainees agreed that non-medical expert ered with one concept. CanMEDS roles are important, they consistently I like that about [the] cases. Like handover expressed dissatisfaction with existing CanMEDS cur- includes…communication, collaboration… ricula, finding the approaches reductionist and artificial. Trainees appreciated incorporation of CanMEDS topics in the VP case, and especially that multiple CanMEDS Category 7. Simulations cannot replace real world experience competencies were introduced without disrupting the of patient care flow of the case. VP cases may provide an exciting new Most trainees felt that, although VP cases provided a arena where CanMEDS competencies can be introduced useful adjunct, many of the CanMEDS competencies are or reinforced. best achieved through real world experience. Jeimy et al. BMC Res Notes (2018) 11:357 Page 6 of 7 Authors’ contributions There are several advantages to integrating VP cases SJ and JW collected, analyzed and interpreted the data. SJ prepared the initial in medical education, including cost benefits, cases that draft of the manuscript. JW and LR were major contributors in developing the closely match real-life situations, the ability to create col- research design, analyzing the data, and critically evaluating the manuscript, and agree to be accountable for all aspects of the work. All authors read and lections of similar cases, seamless integration of Can- approved the final manuscript MEDS competency training, and the ability to create VP cases with which trainees should ideally gain compe- Author details Division of Clinical Immunology and Allergy, Department of General Internal tence. Future work will concentrate case enhancements Medicine, Western University, B3-110, 268 Grosvenor Street, London, ON N6A based on feedback, and cases that can provide real-time 1M1, Canada. Department of General Internal Medicine, University Health feedback or introduce different challenges based on Network - Toronto General Hospital, University of Toronto, EN 14-218, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada. training levels. We hope to create a larger pool of cases to allow for standardization in trainees’ exposure to com- Acknowledgements mon and atypical internal medicine presentations. The authors are grateful to Dr. Michael Li for his contributions to the creation of the virtual patient cases. Competing interests Limitations The authors declare that they have no competing interests. Our analysis is limited by a small sample size and selec- Availability of data and materials tive participation. Another limitation is the use of The datasets generated and analyzed during the current study are available self-assessment to evaluate changes in knowledge and from the corresponding author on reasonable request. confidence in managing UGIB, although analysis of Consent for publication objective knowledge scores corroborated subjective Not applicable. reports. Lastly, our study was limited to trainees at Uni- versity of Toronto, and used one VP case to extrapolate Ethics approval and consent to participate The methods outlined were approved by the institutional research ethics data. It would be interesting to evaluate whether our data board at the University of Toronto. Written consent was obtained from all is reproducible for different VP cases, and at other medi - study participants. cal training programs. Funding None to declare. Additional files Publisher’s Note Additional file 1: Additional Questionaries. Assessment of participant Springer Nature remains neutral with regard to jurisdictional claims in pub- characteristics, self-evaluation, knowledge assessment, and case-based lished maps and institutional affiliations. feedback. Questionnaires developed for the study, to assess participant demographics, perceived confidence in diagnostic and management Received: 13 April 2018 Accepted: 1 June 2018 abilities, knowledge, and feedback on the virtual patient case. Additional file 2: Table S1. Change in Level of Confidence after Inter - vention (Median Change in Likert Scale Rating [IQR1, IQR3]). Trainees’ perceived self-confidence in diagnostic and management abilities, measured on a 5-point rating scale ranging from 1 (“poor” confidence) to References 5 (“excellent” confidence). Median change before and after completing 1. Reed DA, Levine RB, Miller RG, Ashar BH, Bass EB, Rice TN, Cofrancesco J the virtual patient case are reported. Jr. Eec ff t of residency duty-hour limits: views of key clinical faculty. Arch Intern Med. 2007;167:1487–92. Additional file 3: Table S2. Participant Characteristics for Participants 2. Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: who Completed Pre-test Only vs. Pre- and Post- Test (% of Participants in an ethical imperative. Acad Med. 2003;78:783–8. Each Intervention Group). Demographic characteristics and perceived 3. McGee JB, Neill J, Goldman L, Casey E. Using multimedia virtual patients confidence of participants who completed the pre-test only, with those to enhance the clinical curriculum for medical students. Stud Health who completed both the pre- and post- tests, to confirm that these Technol Inform. 1998;52(Pt 2):732–5. groups were not significantly different. 4. Simo A, Cavazza M, Kijima R. Virtual patients in clinical medicine. Stud Additional file 4: Table S3. Participant Case Evaluation (% of Participants Health Technol Inform. 2004;98:353–9. in Each Intervention Group). Trainees’ evaluation of the virtual patient case, 5. Vincent C, Young M, Phillips A. Why do people sue doctors? A study of based on questionnaires developed for the study. patients and relatives taking legal action. Lancet. 1994;343:1609–13. Additional file 5: Table S4. Themes in Free-Text Feedback. Trainees’ free- 6. Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes- text evaluation of the virtual patient case. based framework of physician competencies. Med Teach. 2007;29:642–7. 7. Chou S, Cole G, McLaughlin K, Lockyer J. CanMEDS evaluation in Cana- Additional file 6: Table S5. Focus group participant characteristics. dian postgraduate training programmes: tools used and programme Demographic features of trainees who participated in focus groups. director satisfaction. Med Educ. 2008;42:879–86. 8. Ritchie J, Spencer L. Analyzing Qualitative Data. In: Bryman A, Burgess RG, editors. Qualitative data analysis for applied policy research. London: Routledge; 1994. p. 173–94. Abbreviations 9. Cave J, Goldacre M, Lambert T, Woolf K, Jones A, Dacre J. Newly qualified VP: virtual patient; CanMEDS: Canadian Medical Education Directives for doctors’ views about whether their medical school had trained them Specialists; UGIB: upper gastrointestinal bleed. well: questionnaire surveys. BMC Med Educ. 2007;7:38. Jeimy et al. BMC Res Notes (2018) 11:357 Page 7 of 7 10. Goldacre MJ, Davidson JM, Lambert TW. Doctors’ views of their first year 14. Issenberg SB, McGaghie WC, Petrusa ER, Lee GD, Scalese RJ. Features and of medical work and postgraduate training in the UK: questionnaire uses of high-fidelity medical simulations that lead to effective learning: a surveys. Med Educ. 2003;37:802–8. BEME systematic review. Med Teach. 2005;27:10–28. 11. Ochsmann EB, Zier U, Drexler H, Schmid K. Well prepared for work? 15. Davis D, Bordage G, Moores LK, Bennett N, Marinopoulos SS, Mazmanian Junior doctors’ self-assessment after medical education. BMC Med Educ. PE, Dorman T, McCrory D. The science of continuing medical education: 2011;11:99. terms, tools, and gaps: effectiveness of continuing medical education: 12. Cook DA, Hamstra SJ, Brydges R, Zendejas B, Szostek JH, Wang AT, Erwin American College of Chest Physicians Evidence-Based Educational PJ, Hatala R. Comparative effectiveness of instructional design features in Guidelines. Chest. 2009;135:8S–16S. simulation-based education: systematic review and meta-analysis. Med Teach. 2013;35:e867–98. 13. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA. 2008;300:1181–96. Ready to submit your research ? Choose BMC and benefit from: fast, convenient online submission thorough peer review by experienced researchers in your field rapid publication on acceptance support for research data, including large and complex data types • gold Open Access which fosters wider collaboration and increased citations maximum visibility for your research: over 100M website views per year At BMC, research is always in progress. Learn more biomedcentral.com/submissions

Journal

BMC Research NotesSpringer Journals

Published: Jun 5, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off