Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Use of the Smartphone App WhatsApp as an E-Learning Method for Medical Residents: Multicenter Controlled Randomized Trial

Use of the Smartphone App WhatsApp as an E-Learning Method for Medical Residents: Multicenter... Background: The WhatsApp smartphone app is the most widely used instant messaging app in the world. Recent studies reported the use of WhatsApp for educational purposes, but there is no prospective study comparing WhatsApp’s pedagogical effectiveness to that of any other teaching modality. Objective: The main objective of this study was to measure the impact of a learning program via WhatsApp on clinical reasoning in medical residents. Methods: This prospective, randomized, multicenter study was conducted among first- and second-year anesthesiology residents (offline recruitment) from four university hospitals in France. Residents were randomized in two groups of online teaching (WhatsApp and control). The WhatsApp group benefited from daily delivery of teaching documents on the WhatsApp app and a weekly clinical case supervised by a senior physician. In the control group, residents had access to the same documents via a traditional computer electronic learning (e-learning) platform. Medical reasoning was self-assessed online by a script concordance test (SCT; primary parameter), and medical knowledge was assessed using multiple-choice questions (MCQs). The residents also completed an online satisfaction questionnaire. Results: In this study, 62 residents were randomized (32 to the WhatsApp group and 30 to the control group) and 22 residents in each group answered the online final evaluation. We found a difference between the WhatsApp and control groups for SCTs (60% [SD 9%] vs 68% [SD 11%]; P=.006) but no difference for MCQs (18/30 [SD 4] vs 16/30 [SD 4]; P=.22). Concerning satisfaction, there was a better global satisfaction rate in the WhatsApp group than in the control group (8/10 [interquartile range 8-9] vs 8/10 [interquartile range 8-8]; P=.049). Conclusions: Compared to traditional e-learning, the use of WhatsApp for teaching residents was associated with worse clinical reasoning despite better global appreciation. The use of WhatsApp probably contributes to the dispersion of attention linked to the use of the smartphone. The impact of smartphones on clinical reasoning should be studied further. (JMIR Mhealth Uhealth 2019;7(4):e12825) doi: 10.2196/12825 KEYWORDS education, medical, graduate/methods; educational measurement; anesthesiology; internship and residency; trauma; hemorrhage; mobile applications; WhatsApp; smartphone; teaching materials; mobile phone http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 1 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al from four French university hospitals with trauma center Introduction (Amiens, Caen, Lille, and Rouen) comparing WhatsApp to a control group. Since computer-based e-learning did not show Many computer-based teaching materials have been developed any noninferiority compared to traditional teaching, it was in recent years, and electronic learning (e-learning) is becoming chosen as a control teaching platform for this study [16,17]. increasingly popular in medical schools, with the appearance The Ethics and Evaluation Committee for Non-Interventional of guides on e-learning deployment [1]. E-learning has many Research of Rouen University Hospital approved the study organizational advantages over face-to-face teaching: temporal (E2017-37). All participants received information before any and spatial flexibility for learners, live updates, and easy and study procedures were undertaken, and residents were invited uniform dissemination of teaching resources for teachers. to participate as subjects in the study. All information about Moreover, the emergence of social networks facilitates personal and during the trial was sent by email, and participants knew and professional communication and exchange [2,3]. The time that the WhatsApp group was the “intervention” group. spent on mobile phone screens per day (“screen time”) has Agreement to participate was provided online by email or increased exponentially since the introduction of the telephone by each resident who could stop participating at any latest-generation phones known as smartphones, in particular, time. among young people, leading to a growing interest of mobile learning (m-learning) among teachers [1,4-6]. The inclusion criteria were ongoing medical residency, possession of a mobile phone that could download the The WhatsApp smartphone app, developed by WhatsApp Inc WhatsApp app, and attestation for agreement to use WhatsApp (owned by Facebook Inc, Menlo Park, CA), is the most widely for this study. Noninclusion criteria were refusal to participate, used instant messaging app in the world, with more than one noncompatibility of a mobile phone to download WhatsApp, billion active users per month and more than 40 billion and failure to download WhatsApp. This study was carried out WhatsApp messages exchanged each day in 2016 [7]. It allows in addition to the official teaching program of the residents and communication between group participants without the need was not integrated into usual teaching nor did it replace previous for unity in place or time. Participants are free to choose when teaching. they want to access the information posted and can view and interact with other group members regarding the information The primary measure was medical reasoning evaluated by the delivered at any time. In view of its popularity with medical SCT. The secondary parameters were medical knowledge students, it seems interesting to envisage a new use of measured by multiple-choice questions (MCQ); feasibility and WhatsApp, by orienting it toward an educational objective (with acceptability of using WhatsApp, assessed by collecting resident the opportunity to recover screen time from students) [8,9]. The testimonials; the Cronbach coefficient alpha, calculated after first reports of the use of this app for educational purposes date optimizing the test for SCT [18]; and self-assessment (by to the early 2017, for teaching medical students or training quantitative and semiquantitative numerical scales using a pathology residents [10,11]. Both of these observational studies satisfaction questionnaire) of time spent working, quality of showed satisfaction among WhatsApp participants and teaching, global satisfaction, teachers’ availability, impression highlighted the ease of use and the quick access to lessons that the teaching met the learning objectives, relevance of through the app. However, there is no prospective study clinical cases, and volume of teaching documents used. comparing WhatsApp to any other teaching modality. Study Procedures Residents involved in tutored practice exchange groups have After inclusion, residents were randomized with their last names better medical reasoning, as evaluated by the script concordance in two groups (WhatsApp and control) by author TC, using an test (SCT), which is a well-validated medical online open-access app for stratification according to the reasoning–assessment tool for residents [12-14]. Similar to student’s hospital and year of residency [19]. Concerning practice exchange groups, WhatsApp allows direct intervention or evaluation, this was a purely app- or Web-based communication between teachers and students with the trial without face-to-face components between resident and possibility of discussing real clinical cases and commenting on teachers. After randomization and before the beginning of the residents’ management of the case. Thus, we hypothesized that course, all residents were emailed a short evaluation with 10 WhatsApp could have the same effect as practice exchange SCTs and 10 MCQs on basic knowledge of anesthesiology groups on clinical reasoning. The main objective of this study (intensive care, regional anesthesia, obstetrics, etc) to check the was to measure the impact of WhatsApp on clinical reasoning initial comparability of the groups and to familiarize first-year by using the SCT. As severe trauma is one of the leading causes residents with the SCT. Students returned the SCTs and MCQs of death in the world, with more than 5 million deaths, and by email after completing them. After this short evaluation, the posttraumatic hemorrhage is the leading cause of mortality, we WhatsApp group benefited from daily delivery of teaching selected posttraumatic hemorrhage management as the topic documents specially prepared for easy readability on a for our teaching and evaluation [15]. smartphone (from Monday to Thursday, morning and afternoon; 2-4 documents/day; Multimedia Appendix 1) through the Methods WhatsApp app. These documents were inspired by the most recent guidelines on the management of traumatic hemorrhagic Population Selection shock and were validated by anesthesiology teachers (VC, BV, This prospective, randomized, unblinded, multicenter study was and BD) [20,21]. It is strongly suggested that resolution of conducted among first- and second-year anesthesiology residents http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 2 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al clinical cases have a significant role in the acquisition of medical The two groups had the same program and learning objectives. reasoning [22,23]. Thus, every Friday, residents were given a Participants did not receive any documents during the weekends “step-by-step” clinical case on WhatsApp for 3-4 hours and were free to stop the courses at any time. The characteristics (Multimedia Appendix 2), supervised by a senior of WhatsApp-assisted m-learning and traditional e-learning anesthesiologist (TC) who questioned the residents (to create used in this study are summarized in Table 1. At the end of the an interest in the clinical cases) and provided them with teaching period, the two groups had the same formative feedback and validation or correction, if necessary, as described evaluation by 29 SCTs and 30 MCQs sent by email and in the practice exchange groups [12]. Several screenshots of the completed during the month following the end of the teaching use of WhatsApp for learning purposes during the protocol are period (Multimedia Appendices 4 and 5). Students returned the presented in the Multimedia Appendix 3. The total duration of SCTs and MCQs by email after completing them. In case of teaching was 3 weeks, and the choice of the length of the nonresponse, residents were sent two reminders by email before teaching period was based on both the availability of teachers being considered lost to follow-up. The residents of the two and the estimated acceptability of students. In the control group, groups who responded to the final evaluation completed an residents had access to the same documents via a computer online satisfaction questionnaire specifically created for this e-learning platform, and the senior anesthesiologist teacher was study (not previously validated in the literature; Multimedia available by email. They had access to the three clinical cases Appendix 6). with their answers but had no live interaction with a teacher. Table 1. Characteristics of WhatsApp-assisted m-learning and control e-learning used in this study. The two groups had the same program, learning objectives, and educational documents. a b Characteristics WhatsApp group (m-learning ) Control group (traditional e-learning ) Length of teaching 3 weeks 3 weeks Accessibility of educational Sent daily on WhatsApp from Monday to Friday Available on a computer e-learning platform documents Teacher availability Available and can be contacted by WhatsApp Available and can be contacted by email Conduct of clinical cases Live on Friday on WhatsApp, with questions and answers Cases accessible on the platform with their answers. from the teacher as the case progresses Teacher available if the student has any questions. m-learning: mobile learning. e-learning: electronic learning. based on the principle that any answer given by one expert had Design of the Script Concordance Test and an intrinsic value, even if that answer did not coincide with Multiple-Choice Questions those of other experts. In the present study, a group of 13 The MCQs and SCTs were written by one of the teachers (TC). anesthesiologist practitioners regularly involved in the They were directly related to issues covered during teaching management of traumatic hemorrhagic shock formed the expert and were reviewed (and possibly modified, if needed) by two panel. The principles of SCT are that for each item, the answer other teachers (JR and VC). The SCTs were designed as entitled the resident to a credit corresponding to the number of previously described [12,24]. The SCT confronted the residents experts who had chosen it. All items had the same maximum with authentic uncertain clinical situations concerning traumatic credit, and raw scores were transformed proportionally to obtain hemorrhagic shock, which were described in vignettes, each a one-point credit for the answer that was chosen by most corresponding to one of the previously set objectives. The experts. Other choices received a partial credit. Thus, to clinical situations were problematic even for experienced calculate the scores, all results were divided by the number of clinicians, either because there were not enough data or the individuals who had given answers chosen by the largest number situations were ambiguous. There were several options for of respondents. The total score for the test was the sum of all diagnosis, investigation, or treatment. The items (questions) credits earned for each item. The total score was then were based on a panel of questions that an experienced clinician transformed into a percentage score. An automatic correction would consider relevant to this type of clinical setting. The item software (freely accessible on the website of the University of was consistent with the presentation of relevant options and Montreal) was used for scoring [25]. Each MCQ was worth one new data (not described in the vignette). The task for the student point, and it was possible for an MCQ to have several correct was to determine the effect these new data on the status of the answers. To obtain a point for an MCQ, the resident had to tick option. The resident’s task was to assess, using a 5-point Likert all the correct answers and none of the incorrect ones. scale, the influence of this new element on the diagnostic Otherwise, the student did not receive any points. The final hypothesis, the plan for investigation, or the treatment. The rating was based on the total number of proposed MCQs. different points on the scale corresponded to positive values Statistical Analysis (the option was enhanced by the new data), neutral values (the With regard to our previous publication on the use of SCT by data did not change the status of the option), or negative values anesthesiology residents, we assumed that a difference of 6% (this option was ruled out by the data). The scoring system was http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 3 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al between the two groups would be clinically significant [12,24]. Results of the Script Concordance Tests and Based on these findings, assuming that the SD was the same Multiple-Choice Questions between the populations and using a power of 0.90 with a level The lessons took place from March 12 to 30, 2018. For final of statistical significance at .05, it was estimated that 22 students evaluation, SCTs including 12 scenarios for a total of 36 items should be analyzed in each group. A randomized study on were submitted to a panel of 13 experts. Thereafter, 7 items of e-learning showed that about a quarter of the students included the SCT were excluded (not enough variability in replies), do not participate or are lost to follow-up [26]. Based on these leaving 29 items of SCT spread over 12 clinical situations. findings, it was estimated that a minimum of 28 students should According to the recommendations of Lubarsky et al, we be included in each group to be able to analyze 22 students. optimized SCT by performing a post-hoc analysis [18]. Items with high variability, low variability, or binomial responses The values are presented as number and percentage values for were excluded. We obtained a final version with 10 scenarios qualitative variables, as mean and SDs for quantitative variables and 24 items. After this optimization, Cronbach coefficient with a normal distribution, and as median and interquartile range alpha was .55. In the WhatsApp group, 20 residents answered for quantitative variables with a non-normal distribution. the preliminary evaluation, 1 resident who responded to the Residents who did not respond to the final evaluation were preliminary evaluation did not answer the final evaluation, and excluded from the final analysis (lack of analyzable parameters). 3 residents who did not respond to the preliminary evaluation After performing a Shapiro-Wilk normality test, the quantitative answered the final evaluation. In the control group, 22 residents variables were compared using a Student t test (if the distribution answered the preliminary evaluation, 1 resident who responded was normal) or a Mann-Whitney test (if the distribution was to the preliminary evaluation did not answer the final evaluation, not normal). The qualitative variables were analyzed using a and 1 resident who did not answer the preliminary evaluation Fischer or a chi-square test. The significance threshold was set answered the final evaluation. There was no demographic at .05. All statistics were analyzed using GraphPad PRISM disparity between the residents who answered and those who software (v 5.0; GraphPad Software Inc, San Diego, CA). did not answer the final evaluation. Their main characteristics are summarized in Tables 3 and 4. The flow chart of the study Results is presented in Figure 1. Residents’ Characteristics On the preliminary evaluation (before teaching), there was no Among 142 eligible anesthesiology residents, 62 (44%) agreed significant difference between the WhatsApp and control groups to participate and were randomized as follows: 32 to the for SCT (64% [SD 7%] vs 62% [SD 6%]; P=.41) or MCQ (8/10 WhatsApp group and 30 to the control group. Their main [SD 1] vs 7/10 [SD 2]; P=.33), showing no difference in clinical characteristics are summarized in Table 2. Two students reasoning or medical knowledge. For the final evaluation (after randomized to the WhatsApp group were excluded after teaching), we found a significant difference between the randomization. The first withdrew from the study for personal WhatsApp and control groups for SCT (60% [SD 9%] vs 68% reasons, and the second was excluded following failure to [SD 11%]; P=.006) but not for MCQs (18/30 [SD 4] vs 16/30 download WhatsApp. [SD 4]; P=.22). In the WhatsApp group, there was no difference in the SCT between the initial evaluation and the final evaluation (P=.14). In the control group, the SCT scores of the final evaluation were significantly higher than those of the initial evaluation (P=.02). Table 2. Demographic characteristics of the residents. Characteristic Control group (n=30), n (%) WhatsApp group (n=32), n (%) Year of residency First 14 (47) 14 (44) Second 16 (53) 18 (56) Sex Male 17 (57) 23 (72) Female 13 (43) 9 (28) University hospital Rouen 10 (33) 11 (34) Lille 10 (33) 11 (34) Caen 6 (20) 5 (16) Amiens 4 (14) 5 (16) http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 4 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al Table 3. Demographic characteristics of residents who answered the final evaluation. Characteristic Control group (n=22), n (%) WhatsApp group (n=22), n (%) Year of residency First 9 (41) 9 (41) Second 13 (59) 13 (59) Sex Male 14 (64) 16 (73) Female 8 (36) 6 (27) University hospital Rouen 10 (45) 10 (45) Lille 6 (27) 5 (23) Caen 5 (23) 4 (18) Amiens 1 (5) 2 (10) Table 4. Demographic characteristics of residents who did not answer the final evaluation. Characteristic Control group (n=8), n (%) WhatsApp group (n=8), n (%) Year of residency First 5 (63) 3 (37) Second 3 (37) 5 (63) Sex Male 3 (37) 5 (63) Female 5 (63) 3 (37) University hospital Rouen 0 (0) 0 (0) Lille 4 (50) 4 (50) Caen 1 (13) 1 (13) Amiens 3 (37) 3 (37) of the perceived quality of educational materials (9/10 Use of WhatsApp and Residents’ Satisfaction [interquartile range 8-10] vs 8/10 [interquartile range 8-10]; The residents of the two groups who filled the final evaluation P=.15), the usefulness and relevance of clinical cases (10/10 were asked to fill an online satisfaction questionnaire. Twenty [interquartile range 8-10] vs 9/10 [interquartile range 7-10]; (67%) residents in the WhatsApp group and 13 (43%) residents P=.40), the quantity of teaching documents used by the residents in the control group answered this questionnaire. All the scores (in the WhatsApp group, 14 residents [70%] used more than from the satisfaction evaluation had a non-normal distribution. 50% of the documents and 6 [30%] used less than 50% of the There was a difference between the WhatsApp and control documents; in the control group, 10 residents [77%] used more groups, with the WhatsApp group showing a better global than 50% of the documents and 3 [23%] used less than 50% of satisfaction rate (8/10 [interquartile range, 8-9] vs 8/10 the documents; P=.66), or the time spent working on the [interquartile range 8-8]; P=.049), a better feeling that the program (in the WhatsApp group, 2 residents [10%] spent lessons met the learning objectives (10/10 [interquartile range between 5 h and 10 h and 18 [90%] spent between 1 h and 5 h; 8-10] vs 8/10 [interquartile range 7-10]; P=.03), and a feeling in the control group, 4 residents [31%] spent between 5 h and that the teachers were more available (10/10 [interquartile range 10 h and 9 [69%] spent between 1 h and 5 h; P=.18). Textbox 9-10] vs 9/10 [interquartile range 8-10]; P=.007). We found no 1 presents quotes from the free comments section of the differences between the WhatsApp and control groups in terms satisfaction questionnaire of the WhatsApp group. http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 5 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al Figure 1. Flowchart of the study. e-learning: electronic learning; MCQ: multiple-choice question; SCT: script concordance test. Textbox 1. Quotes from the free comments section of the satisfaction questionnaire (WhatsApp group residents). “For participation to the Friday clinical case it depends on the availability of everyone. The fact that it is on whatsapp makes it easier to communicate and ask questions. Having notifications is more motivating to consult documents than on a platform.” “Having what's app notifications allows me to be more assiduous, the possibility to ask questions directly in the conversation is a big advantage, it sometimes allows small discussions, so very useful. Great classes, interesting cases, and not feeling evaluated is fun. Suggestion: a new session.” “The documents were very well done, difficult on Friday to answer all the questions of the clinical cases online according to our occupations in the ward.” “Very good idea to teach via WhatsApp, which allows to be informed quickly of the presence of new educational documents and to have regular reminder shots since the notifications are displayed. Doing clinical cases on the application during a day with the participation of several people is very instructive. The only problem is that the documents are difficult to consult on a small telephone, perhaps it would be necessary to adapt the documents in the form of slides format telephone. Otherwise it was great! High quality educational documents. Thank you!” “Interesting to be able to consult documents via whatsapp. As far as Friday clinical cases are concerned, it is quite difficult to switch between ward presence or other obligations and whatsapp.” “Very nice project. I think it's useful to have cards per whatsapp but the flow was too high: 4 documents per day, we end up having too much delay in the readings.” “The idea is good but it's quite laborious to read lessons on a mobile phone, especially long pdf. The well ventilated and clear synthetic dcouments [sic] are on the other hand interesting. It is also interesting to be able to ask questions directly and get quick answers. But it can't replace classical education.” http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 6 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al satisfaction for otorhinolaryngology and head and neck surgery Discussion disorders in undergraduate medical students [28]. Despite the absence of differences in knowledge gain in the mobile Principal Results interactive multimedia group, satisfaction was higher in the This randomized, multicenter study is the first to focus on the mobile group (like in our cohort). Therefore, we can assume impact of WhatsApp on clinical reasoning in medical students. that our data are concordant with the literature on m-learning. We found that the use of WhatsApp, instead of a traditional Finally, the daily use of WhatsApp for medical education e-learning platform, to teach a specific topic was associated probably contributes to the dispersion of attention linked to the with worse clinical reasoning despite better global appreciation. use of the smartphone. In view of these results, it does not seem Comparison With Prior Work justified to continue to develop WhatsApp for teaching medical Several recent studies have reported the potential interest of reasoning to medical residents. However, the targeted use of specific smartphone apps in medical education, but our objective WhatsApp with other educational objectives (eg, medical was to assess the interest of a very widely used nonmedical app imaging or video) remains to be evaluated and should be the (thus easily usable by all) for teaching [27-29]. Given that subject of future randomized studies. It is known that blended WhatsApp allows interaction between teachers and students, learning can have a beneficial effect on knowledge acquisition with the possibility of discussing clinical cases, we believed its in health professions [30]. Thus, it might also be interesting to use would improve medical reasoning, as previously described study the use of WhatsApp as a complement to another form for face-to-face practice exchange groups [12]. We did not find of teaching. Given the increasing use of smartphones by health any difference in the global amount of work or the number of workers, it also seems appropriate to consider future work to educational documents consulted, which is consistent with assess the quality of clinical reasoning between two populations similar personal work between the two groups. It has been of physicians with or without usual smartphone use in hospitals. shown that e-learning methods improve the medical knowledge Limitations of health care professionals [28]. The absence of a difference Our study has several limitations. First, the Cronbach coefficient in the medical knowledge assessed by MCQs shows that the alpha in our SCT evaluation was low. The minimum coefficient weakness of clinical reasoning related to WhatsApp is not usually retained for normative evaluations is 0.7, but in our related to less knowledge of the subject. We can therefore work the evaluation was only formative and integrated into assume that this decrease in the quality of reasoning is directly teaching. The limited number of SCTs probably explains this related to WhatsApp or the use of a smartphone. It is likely that low coefficient. However, teaching in a specific and specialized reading on WhatsApp between two other activities was less area made it difficult to find at least 60 SCTs (as is usually effective than time spent solely on an e-learning platform. A recommended) without redundancy [18]. Second, residents’ recent study showed that a smartphone app dedicated to teaching participation in our work was limited: Only 62 of 142 residents medical students Dermatology, in combination with traditional participated. As previously observed, self-training with teaching, improved medical knowledge measured by MCQs e-learning is impacted by a significant dropout rate [26]. In our [27]. Although we did not find any improvement in medical work, only 22 of the 30 residents participated in the final knowledge in our work, the smartphone was seen as an evaluation. As this teaching was optional, participation in our alternative to conventional e-learning and not as a complement. study represented additional personal work for the residents. It It is interesting to note that in the literature, most of the is therefore possible that the majority of residents were educational benefits reported with smartphone use stem from discouraged by this prospect. In addition, residents without very “visual” specialties (Dermatology or Pathology) and that smartphones or those who did not wish to use WhatsApp this tool, which allows easy communication of iconography, is logically refused to participate. Third, we could not prevent probably more relevant in this context than in “less visual” cross-communication among students while they answered the medical specialties [10,27]. SCTs and MCQs, and the residents could have communicated Residents pointed out two limitations: the difficulty of with each other during the final evaluation. The fact that this participating in clinical cases on Friday in parallel with their evaluation was not sanctioned and had no value, as it was not usual activities and the difficulty in referring to documents on integrated into usual teaching methods, probably limited this small smartphone screens. Unlike for practice exchange groups, communication. Finally, we did not use a prevalidated there was no time dedicated specifically to clinical case questionnaire to measure satisfaction. As we wanted to evaluate resolution on WhatsApp, and residents had to respond in specific points related to the use of WhatsApp in our population, addition to their usual activities [12]. This probably favored a we created a new dedicated questionnaire, but this choice made multitasking activity with a difficulty to focus on the it more difficult to compare our satisfaction results to those of pedagogical content. However, it is interesting to note that the others. comments from WhatsApp residents were very positive, with Conclusions a higher overall satisfaction rating. The novelty and originality of the concept probably contributed to this satisfaction, but it Compared to traditional e-learning, the use of WhatsApp as an underlines the fact that the students were not aware of the m-learning method for residents teaching is associated with possible negative impact of the use of WhatsApp. A recent worse clinical reasoning despite better global appreciation. The randomized pedagogic study assessed the impact of learning use of the WhatsApp app probably contributes to the dispersion modules using m-learning on knowledge gain, skill gain, and of attention linked to the use of the smartphone. http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 7 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al Acknowledgments The authors are grateful to Nikki Sabourin-Gibbs, Rouen University Hospital, for her help in editing the manuscript; to all the senior anesthesiologists from Caen and Rouen University Hospitals who agreed to be experts for SCT answers; and to all the anesthesiology residents who agreed to participate in this work. Funding support was provided solely from departmental sources. Conflicts of Interest None declared. Multimedia Appendix 1 Teaching documents especially prepared for easy readability on a smartphone (in French with English translation). [PDF File (Adobe PDF File), 3MB-Multimedia Appendix 1] Multimedia Appendix 2 "Step-by-step" clinical cases (in French with English translation). [PDF File (Adobe PDF File), 428KB-Multimedia Appendix 2] Multimedia Appendix 3 Several examples of the use of WhatsApp for learning purposes during the protocol (screenshots in French with English translation). [PDF File (Adobe PDF File), 741KB-Multimedia Appendix 3] Multimedia Appendix 4 Script concordance test used for the final evaluation (in French with English translation). [PDF File (Adobe PDF File), 632KB-Multimedia Appendix 4] Multimedia Appendix 5 Multiple-choice questions used for the final evaluation (in French with English translation). [PDF File (Adobe PDF File), 720KB-Multimedia Appendix 5] Multimedia Appendix 6 Online satisfaction questionnaire (in French with English translation). [PDF File (Adobe PDF File), 550KB-Multimedia Appendix 6] Multimedia Appendix 7 CONSORT‐EHEALTH checklist (V 1.6.1). [PDF File (Adobe PDF File), 2MB-Multimedia Appendix 7] References 1. Ellaway R, Masters K. AMEE Guide 32: e-Learning in medical education Part 1: Learning, teaching and assessment. Med Teach 2008 Jun;30(5):455-473. [doi: 10.1080/01421590802108331] [Medline: 18576185] 2. Alsobayel H. Use of Social Media for Professional Development by Health Care Professionals: A Cross-Sectional Web-Based Survey. JMIR Med Educ 2016 Sep 12;2(2):e15 [FREE Full text] [doi: 10.2196/mededu.6232] [Medline: 27731855] 3. Chan WS, Leung AY. Use of Social Network Sites for Communication Among Health Professionals: Systematic Review. J Med Internet Res 2018 Mar 28;20(3):e117. [doi: 10.2196/jmir.8382] [Medline: 29592845] 4. Christensen MA, Bettencourt L, Kaye L, Moturu ST, Nguyen KT, Olgin JE, et al. Direct Measurements of Smartphone Screen-Time: Relationships with Demographics and Sleep. PLoS One 2016;11(11):e0165331 [FREE Full text] [doi: 10.1371/journal.pone.0165331] [Medline: 27829040] 5. Eggermont S, Bloemendaal PM, van Baalen JM. E-learning any time any place anywhere on mobile devices. Perspect Med Educ 2013 Mar 30 [FREE Full text] [doi: 10.1007/s40037-013-0045-4] [Medline: 23670682] http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 8 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al 6. Masters K, Ellaway RH, Topps D, Archibald D, Hogue RJ. Mobile technologies in medical education: AMEE Guide No. 105. Med Teach 2016 Jun;38(6):537-549. [doi: 10.3109/0142159X.2016.1141190] [Medline: 27010681] 7. Meola A. Business Insider French. WhatsApp is the most popular chat app in more than half the world Internet URL: http:/ /www.businessinsider.fr/us/whatsapp-is-the-most-popular-chat-app-in-more-than-half-the-world-2016-5/ [accessed 2018-11-12] [WebCite Cache ID 73sJKGsIb] 8. AlFaris E, Irfan F, Ponnamperuma G, Jamal A, Van der Vleuten C, Al Maflehi N, et al. The pattern of social media use and its association with academic performance among medical students. Med Teach 2018 Dec;40(sup1):S77-S82. [doi: 10.1080/0142159X.2018.1465536] [Medline: 29732945] 9. Jamal A, Temsah M, Khan SA, Al-Eyadhy A, Koppel C, Chiang MF. Mobile Phone Use Among Medical Residents: A Cross-Sectional Multicenter Survey in Saudi Arabia. JMIR Mhealth Uhealth 2016 May 19;4(2):e61 [FREE Full text] [doi: 10.2196/mhealth.4904] [Medline: 27197618] 10. Goyal A, Tanveer N, Sharma P. WhatsApp for Teaching Pathology Postgraduates: A Pilot Study. J Pathol Inform 2017;8:6 [FREE Full text] [doi: 10.4103/2153-3539.201111] [Medline: 28400995] 11. Raiman L, Antbring R, Mahmood A. WhatsApp messenger as a tool to supplement medical education for medical students on clinical attachment. BMC Med Educ 2017 Jan 06;17(1):7 [FREE Full text] [doi: 10.1186/s12909-017-0855-x] [Medline: 28061777] 12. Compère V, Abily J, Moriceau J, Gouin A, Veber B, Dupont H, et al. Residents in tutored practice exchange groups have better medical reasoning as measured by script concordance test: a controlled, nonrandomized study. J Clin Anesth 2016 Aug;32:236-241. [doi: 10.1016/j.jclinane.2016.03.012] [Medline: 27290981] 13. Ducos G, Lejus C, Sztark F, Nathan N, Fourcade O, Tack I, et al. The Script Concordance Test in anesthesiology: Validation of a new tool for assessing clinical reasoning. Anaesth Crit Care Pain Med 2015 Feb;34(1):11-15. [doi: 10.1016/j.accpm.2014.11.001] [Medline: 25829309] 14. Bursztejn A, Cuny J, Adam J, Sido L, Schmutz J, de Korwin J, et al. Usefulness of the script concordance test in dermatology. J Eur Acad Dermatol Venereol 2011 Dec;25(12):1471-1475. [doi: 10.1111/j.1468-3083.2011.04008.x] [Medline: 22077872] 15. Dutton RP, Stansbury LG, Leone S, Kramer E, Hess JR, Scalea TM. Trauma mortality in mature trauma systems: are we doing better? An analysis of trauma mortality patterns, 1997-2008. J Trauma 2010 Sep;69(3):620-626. [doi: 10.1097/TA.0b013e3181bbfe2a] [Medline: 20093983] 16. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA 2008 Sep 10;300(10):1181-1196. [doi: 10.1001/jama.300.10.1181] [Medline: 18780847] 17. Kulier R, Coppus SFPJ, Zamora J, Hadley J, Malick S, Das K, et al. The effectiveness of a clinically integrated e-learning course in evidence-based medicine: a cluster randomised controlled trial. BMC Med Educ 2009 May 12;9:21 [FREE Full text] [doi: 10.1186/1472-6920-9-21] [Medline: 19435520] 18. Lubarsky S, Dory V, Duggan P, Gagnon R, Charlin B. Script concordance testing: from theory to practice: AMEE guide no. 75. Med Teach 2013;35(3):184-193. [doi: 10.3109/0142159X.2013.760036] [Medline: 23360487] 19. Dcode. Tirage au Sort URL: https://www.dcode.fr/tirage-au-sort-nombre-aleatoire [accessed 2018-11-14] [WebCite Cache ID 73vLExugV] 20. Rossaint R, Bouillon B, Cerny V, Coats TJ, Duranteau J, Fernández-Mondéjar E, et al. The European guideline on management of major bleeding and coagulopathy following trauma: fourth edition. Crit Care 2016 Apr 12;20:100 [FREE Full text] [doi: 10.1186/s13054-016-1265-x] [Medline: 27072503] 21. Duranteau J, Asehnoune K, Pierre S, Ozier Y, Leone M, Lefrant J. Recommandations sur la réanimation du choc hémorragique. Anesthésie & Réanimation 2015 Feb;1(1):62-74. [doi: 10.1016/j.anrea.2014.12.007] 22. Kassirer JP. Teaching clinical reasoning: case-based and coached. Acad Med 2010 Jul;85(7):1118-1124. [Medline: 20603909] 23. Schmidt HG, Mamede S. How to improve the teaching of clinical reasoning: a narrative review and a proposal. Med Educ 2015 Oct;49(10):961-973. [doi: 10.1111/medu.12775] [Medline: 26383068] 24. Enser M, Moriceau J, Abily J, Damm C, Occhiali E, Besnier E, et al. Background noise lowers the performance of anaesthesiology residents' clinical reasoning when measured by script concordance: A randomised crossover volunteer study. Eur J Anaesthesiol 2017 Dec;34(7):464-470. [doi: 10.1097/EJA.0000000000000624] [Medline: 28394819] 25. Faculté de Médecine - Université de Montréal Internet. Obtenir les scores de vos TCS par voie informatique - Centre de pédagogie appliquée aux sciences de la santé (CPASS) URL: https://www.cpass.umontreal.ca/recherche/ groupe-de-recherche-cpass/axes-de-recherches/concordance/tcs/corriger_tcs/ [accessed 2018-11-14] [WebCite Cache ID 73vBHLU6Q] 26. Moreira IC, Ventura SR, Ramos I, Rodrigues PP. Development and assessment of an e-learning course on breast imaging for radiographers: a stratified randomized controlled trial. J Med Internet Res 2015;17(1):e3 [FREE Full text] [doi: 10.2196/jmir.3344] [Medline: 25560547] 27. Fransen F, Martens H, Nagtzaam I, Heeneman S. Use of e-learning in clinical clerkships: effects on acquisition of dermatological knowledge and learning processes. Int J Med Educ 2018 Jan 17;9:11-17 [FREE Full text] [doi: 10.5116/ijme.5a47.8ab0] [Medline: 29352748] http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 9 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al 28. Lee L, Chao Y, Huang C, Fang J, Wang S, Chuang C, et al. Cognitive Style and Mobile E-Learning in Emergent Otorhinolaryngology-Head and Neck Surgery Disorders for Millennial Undergraduate Medical Students: Randomized Controlled Trial. J Med Internet Res 2018 Feb 13;20(2):e56 [FREE Full text] [doi: 10.2196/jmir.8987] [Medline: 29439943] 29. Shaw CM, Tan SA. Integration of mobile technology in educational materials improves participation: creation of a novel smartphone application for resident education. J Surg Educ 2015;72(4):670-673. [doi: 10.1016/j.jsurg.2015.01.015] [Medline: 25823746] 30. Liu Q, Peng W, Zhang F, Hu R, Li Y, Yan W. The Effectiveness of Blended Learning in Health Professions: Systematic Review and Meta-Analysis. J Med Internet Res 2016;18(1):e2 [FREE Full text] [doi: 10.2196/jmir.4807] [Medline: 26729058] Abbreviations e-learning: electronic learning MCQ: multiple-choice questions m-learning: mobile learning SCT: script concordance test Edited by G Eysenbach; submitted 15.11.18; peer-reviewed by LA Lee, VK Bandi, A Paglialonga; comments to author 10.01.19; revised version received 17.01.19; accepted 23.02.19; published 09.04.19 Please cite as: Clavier T, Ramen J, Dureuil B, Veber B, Hanouz JL, Dupont H, Lebuffe G, Besnier E, Compere V JMIR Mhealth Uhealth 2019;7(4):e12825 URL: http://mhealth.jmir.org/2019/4/e12825/ doi: 10.2196/12825 PMID: 30964435 ©Thomas Clavier, Julie Ramen, Bertrand Dureuil, Benoit Veber, Jean-Luc Hanouz, Hervé Dupont, Gilles Lebuffe, Emmanuel Besnier, Vincent Compere. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 09.04.2019. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included. http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 10 (page number not for citation purposes) XSL FO RenderX http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png JMIR mHealth and uHealth JMIR Publications

Use of the Smartphone App WhatsApp as an E-Learning Method for Medical Residents: Multicenter Controlled Randomized Trial

Loading next page...
 
/lp/jmir-publications/use-of-the-smartphone-app-whatsapp-as-an-e-learning-method-for-medical-ph0JxICrjO

References (35)

Publisher
JMIR Publications
Copyright
Copyright © The Author(s). Licensed under Creative Commons Attribution cc-by 4.0
ISSN
2291-5222
DOI
10.2196/12825
Publisher site
See Article on Publisher Site

Abstract

Background: The WhatsApp smartphone app is the most widely used instant messaging app in the world. Recent studies reported the use of WhatsApp for educational purposes, but there is no prospective study comparing WhatsApp’s pedagogical effectiveness to that of any other teaching modality. Objective: The main objective of this study was to measure the impact of a learning program via WhatsApp on clinical reasoning in medical residents. Methods: This prospective, randomized, multicenter study was conducted among first- and second-year anesthesiology residents (offline recruitment) from four university hospitals in France. Residents were randomized in two groups of online teaching (WhatsApp and control). The WhatsApp group benefited from daily delivery of teaching documents on the WhatsApp app and a weekly clinical case supervised by a senior physician. In the control group, residents had access to the same documents via a traditional computer electronic learning (e-learning) platform. Medical reasoning was self-assessed online by a script concordance test (SCT; primary parameter), and medical knowledge was assessed using multiple-choice questions (MCQs). The residents also completed an online satisfaction questionnaire. Results: In this study, 62 residents were randomized (32 to the WhatsApp group and 30 to the control group) and 22 residents in each group answered the online final evaluation. We found a difference between the WhatsApp and control groups for SCTs (60% [SD 9%] vs 68% [SD 11%]; P=.006) but no difference for MCQs (18/30 [SD 4] vs 16/30 [SD 4]; P=.22). Concerning satisfaction, there was a better global satisfaction rate in the WhatsApp group than in the control group (8/10 [interquartile range 8-9] vs 8/10 [interquartile range 8-8]; P=.049). Conclusions: Compared to traditional e-learning, the use of WhatsApp for teaching residents was associated with worse clinical reasoning despite better global appreciation. The use of WhatsApp probably contributes to the dispersion of attention linked to the use of the smartphone. The impact of smartphones on clinical reasoning should be studied further. (JMIR Mhealth Uhealth 2019;7(4):e12825) doi: 10.2196/12825 KEYWORDS education, medical, graduate/methods; educational measurement; anesthesiology; internship and residency; trauma; hemorrhage; mobile applications; WhatsApp; smartphone; teaching materials; mobile phone http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 1 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al from four French university hospitals with trauma center Introduction (Amiens, Caen, Lille, and Rouen) comparing WhatsApp to a control group. Since computer-based e-learning did not show Many computer-based teaching materials have been developed any noninferiority compared to traditional teaching, it was in recent years, and electronic learning (e-learning) is becoming chosen as a control teaching platform for this study [16,17]. increasingly popular in medical schools, with the appearance The Ethics and Evaluation Committee for Non-Interventional of guides on e-learning deployment [1]. E-learning has many Research of Rouen University Hospital approved the study organizational advantages over face-to-face teaching: temporal (E2017-37). All participants received information before any and spatial flexibility for learners, live updates, and easy and study procedures were undertaken, and residents were invited uniform dissemination of teaching resources for teachers. to participate as subjects in the study. All information about Moreover, the emergence of social networks facilitates personal and during the trial was sent by email, and participants knew and professional communication and exchange [2,3]. The time that the WhatsApp group was the “intervention” group. spent on mobile phone screens per day (“screen time”) has Agreement to participate was provided online by email or increased exponentially since the introduction of the telephone by each resident who could stop participating at any latest-generation phones known as smartphones, in particular, time. among young people, leading to a growing interest of mobile learning (m-learning) among teachers [1,4-6]. The inclusion criteria were ongoing medical residency, possession of a mobile phone that could download the The WhatsApp smartphone app, developed by WhatsApp Inc WhatsApp app, and attestation for agreement to use WhatsApp (owned by Facebook Inc, Menlo Park, CA), is the most widely for this study. Noninclusion criteria were refusal to participate, used instant messaging app in the world, with more than one noncompatibility of a mobile phone to download WhatsApp, billion active users per month and more than 40 billion and failure to download WhatsApp. This study was carried out WhatsApp messages exchanged each day in 2016 [7]. It allows in addition to the official teaching program of the residents and communication between group participants without the need was not integrated into usual teaching nor did it replace previous for unity in place or time. Participants are free to choose when teaching. they want to access the information posted and can view and interact with other group members regarding the information The primary measure was medical reasoning evaluated by the delivered at any time. In view of its popularity with medical SCT. The secondary parameters were medical knowledge students, it seems interesting to envisage a new use of measured by multiple-choice questions (MCQ); feasibility and WhatsApp, by orienting it toward an educational objective (with acceptability of using WhatsApp, assessed by collecting resident the opportunity to recover screen time from students) [8,9]. The testimonials; the Cronbach coefficient alpha, calculated after first reports of the use of this app for educational purposes date optimizing the test for SCT [18]; and self-assessment (by to the early 2017, for teaching medical students or training quantitative and semiquantitative numerical scales using a pathology residents [10,11]. Both of these observational studies satisfaction questionnaire) of time spent working, quality of showed satisfaction among WhatsApp participants and teaching, global satisfaction, teachers’ availability, impression highlighted the ease of use and the quick access to lessons that the teaching met the learning objectives, relevance of through the app. However, there is no prospective study clinical cases, and volume of teaching documents used. comparing WhatsApp to any other teaching modality. Study Procedures Residents involved in tutored practice exchange groups have After inclusion, residents were randomized with their last names better medical reasoning, as evaluated by the script concordance in two groups (WhatsApp and control) by author TC, using an test (SCT), which is a well-validated medical online open-access app for stratification according to the reasoning–assessment tool for residents [12-14]. Similar to student’s hospital and year of residency [19]. Concerning practice exchange groups, WhatsApp allows direct intervention or evaluation, this was a purely app- or Web-based communication between teachers and students with the trial without face-to-face components between resident and possibility of discussing real clinical cases and commenting on teachers. After randomization and before the beginning of the residents’ management of the case. Thus, we hypothesized that course, all residents were emailed a short evaluation with 10 WhatsApp could have the same effect as practice exchange SCTs and 10 MCQs on basic knowledge of anesthesiology groups on clinical reasoning. The main objective of this study (intensive care, regional anesthesia, obstetrics, etc) to check the was to measure the impact of WhatsApp on clinical reasoning initial comparability of the groups and to familiarize first-year by using the SCT. As severe trauma is one of the leading causes residents with the SCT. Students returned the SCTs and MCQs of death in the world, with more than 5 million deaths, and by email after completing them. After this short evaluation, the posttraumatic hemorrhage is the leading cause of mortality, we WhatsApp group benefited from daily delivery of teaching selected posttraumatic hemorrhage management as the topic documents specially prepared for easy readability on a for our teaching and evaluation [15]. smartphone (from Monday to Thursday, morning and afternoon; 2-4 documents/day; Multimedia Appendix 1) through the Methods WhatsApp app. These documents were inspired by the most recent guidelines on the management of traumatic hemorrhagic Population Selection shock and were validated by anesthesiology teachers (VC, BV, This prospective, randomized, unblinded, multicenter study was and BD) [20,21]. It is strongly suggested that resolution of conducted among first- and second-year anesthesiology residents http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 2 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al clinical cases have a significant role in the acquisition of medical The two groups had the same program and learning objectives. reasoning [22,23]. Thus, every Friday, residents were given a Participants did not receive any documents during the weekends “step-by-step” clinical case on WhatsApp for 3-4 hours and were free to stop the courses at any time. The characteristics (Multimedia Appendix 2), supervised by a senior of WhatsApp-assisted m-learning and traditional e-learning anesthesiologist (TC) who questioned the residents (to create used in this study are summarized in Table 1. At the end of the an interest in the clinical cases) and provided them with teaching period, the two groups had the same formative feedback and validation or correction, if necessary, as described evaluation by 29 SCTs and 30 MCQs sent by email and in the practice exchange groups [12]. Several screenshots of the completed during the month following the end of the teaching use of WhatsApp for learning purposes during the protocol are period (Multimedia Appendices 4 and 5). Students returned the presented in the Multimedia Appendix 3. The total duration of SCTs and MCQs by email after completing them. In case of teaching was 3 weeks, and the choice of the length of the nonresponse, residents were sent two reminders by email before teaching period was based on both the availability of teachers being considered lost to follow-up. The residents of the two and the estimated acceptability of students. In the control group, groups who responded to the final evaluation completed an residents had access to the same documents via a computer online satisfaction questionnaire specifically created for this e-learning platform, and the senior anesthesiologist teacher was study (not previously validated in the literature; Multimedia available by email. They had access to the three clinical cases Appendix 6). with their answers but had no live interaction with a teacher. Table 1. Characteristics of WhatsApp-assisted m-learning and control e-learning used in this study. The two groups had the same program, learning objectives, and educational documents. a b Characteristics WhatsApp group (m-learning ) Control group (traditional e-learning ) Length of teaching 3 weeks 3 weeks Accessibility of educational Sent daily on WhatsApp from Monday to Friday Available on a computer e-learning platform documents Teacher availability Available and can be contacted by WhatsApp Available and can be contacted by email Conduct of clinical cases Live on Friday on WhatsApp, with questions and answers Cases accessible on the platform with their answers. from the teacher as the case progresses Teacher available if the student has any questions. m-learning: mobile learning. e-learning: electronic learning. based on the principle that any answer given by one expert had Design of the Script Concordance Test and an intrinsic value, even if that answer did not coincide with Multiple-Choice Questions those of other experts. In the present study, a group of 13 The MCQs and SCTs were written by one of the teachers (TC). anesthesiologist practitioners regularly involved in the They were directly related to issues covered during teaching management of traumatic hemorrhagic shock formed the expert and were reviewed (and possibly modified, if needed) by two panel. The principles of SCT are that for each item, the answer other teachers (JR and VC). The SCTs were designed as entitled the resident to a credit corresponding to the number of previously described [12,24]. The SCT confronted the residents experts who had chosen it. All items had the same maximum with authentic uncertain clinical situations concerning traumatic credit, and raw scores were transformed proportionally to obtain hemorrhagic shock, which were described in vignettes, each a one-point credit for the answer that was chosen by most corresponding to one of the previously set objectives. The experts. Other choices received a partial credit. Thus, to clinical situations were problematic even for experienced calculate the scores, all results were divided by the number of clinicians, either because there were not enough data or the individuals who had given answers chosen by the largest number situations were ambiguous. There were several options for of respondents. The total score for the test was the sum of all diagnosis, investigation, or treatment. The items (questions) credits earned for each item. The total score was then were based on a panel of questions that an experienced clinician transformed into a percentage score. An automatic correction would consider relevant to this type of clinical setting. The item software (freely accessible on the website of the University of was consistent with the presentation of relevant options and Montreal) was used for scoring [25]. Each MCQ was worth one new data (not described in the vignette). The task for the student point, and it was possible for an MCQ to have several correct was to determine the effect these new data on the status of the answers. To obtain a point for an MCQ, the resident had to tick option. The resident’s task was to assess, using a 5-point Likert all the correct answers and none of the incorrect ones. scale, the influence of this new element on the diagnostic Otherwise, the student did not receive any points. The final hypothesis, the plan for investigation, or the treatment. The rating was based on the total number of proposed MCQs. different points on the scale corresponded to positive values Statistical Analysis (the option was enhanced by the new data), neutral values (the With regard to our previous publication on the use of SCT by data did not change the status of the option), or negative values anesthesiology residents, we assumed that a difference of 6% (this option was ruled out by the data). The scoring system was http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 3 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al between the two groups would be clinically significant [12,24]. Results of the Script Concordance Tests and Based on these findings, assuming that the SD was the same Multiple-Choice Questions between the populations and using a power of 0.90 with a level The lessons took place from March 12 to 30, 2018. For final of statistical significance at .05, it was estimated that 22 students evaluation, SCTs including 12 scenarios for a total of 36 items should be analyzed in each group. A randomized study on were submitted to a panel of 13 experts. Thereafter, 7 items of e-learning showed that about a quarter of the students included the SCT were excluded (not enough variability in replies), do not participate or are lost to follow-up [26]. Based on these leaving 29 items of SCT spread over 12 clinical situations. findings, it was estimated that a minimum of 28 students should According to the recommendations of Lubarsky et al, we be included in each group to be able to analyze 22 students. optimized SCT by performing a post-hoc analysis [18]. Items with high variability, low variability, or binomial responses The values are presented as number and percentage values for were excluded. We obtained a final version with 10 scenarios qualitative variables, as mean and SDs for quantitative variables and 24 items. After this optimization, Cronbach coefficient with a normal distribution, and as median and interquartile range alpha was .55. In the WhatsApp group, 20 residents answered for quantitative variables with a non-normal distribution. the preliminary evaluation, 1 resident who responded to the Residents who did not respond to the final evaluation were preliminary evaluation did not answer the final evaluation, and excluded from the final analysis (lack of analyzable parameters). 3 residents who did not respond to the preliminary evaluation After performing a Shapiro-Wilk normality test, the quantitative answered the final evaluation. In the control group, 22 residents variables were compared using a Student t test (if the distribution answered the preliminary evaluation, 1 resident who responded was normal) or a Mann-Whitney test (if the distribution was to the preliminary evaluation did not answer the final evaluation, not normal). The qualitative variables were analyzed using a and 1 resident who did not answer the preliminary evaluation Fischer or a chi-square test. The significance threshold was set answered the final evaluation. There was no demographic at .05. All statistics were analyzed using GraphPad PRISM disparity between the residents who answered and those who software (v 5.0; GraphPad Software Inc, San Diego, CA). did not answer the final evaluation. Their main characteristics are summarized in Tables 3 and 4. The flow chart of the study Results is presented in Figure 1. Residents’ Characteristics On the preliminary evaluation (before teaching), there was no Among 142 eligible anesthesiology residents, 62 (44%) agreed significant difference between the WhatsApp and control groups to participate and were randomized as follows: 32 to the for SCT (64% [SD 7%] vs 62% [SD 6%]; P=.41) or MCQ (8/10 WhatsApp group and 30 to the control group. Their main [SD 1] vs 7/10 [SD 2]; P=.33), showing no difference in clinical characteristics are summarized in Table 2. Two students reasoning or medical knowledge. For the final evaluation (after randomized to the WhatsApp group were excluded after teaching), we found a significant difference between the randomization. The first withdrew from the study for personal WhatsApp and control groups for SCT (60% [SD 9%] vs 68% reasons, and the second was excluded following failure to [SD 11%]; P=.006) but not for MCQs (18/30 [SD 4] vs 16/30 download WhatsApp. [SD 4]; P=.22). In the WhatsApp group, there was no difference in the SCT between the initial evaluation and the final evaluation (P=.14). In the control group, the SCT scores of the final evaluation were significantly higher than those of the initial evaluation (P=.02). Table 2. Demographic characteristics of the residents. Characteristic Control group (n=30), n (%) WhatsApp group (n=32), n (%) Year of residency First 14 (47) 14 (44) Second 16 (53) 18 (56) Sex Male 17 (57) 23 (72) Female 13 (43) 9 (28) University hospital Rouen 10 (33) 11 (34) Lille 10 (33) 11 (34) Caen 6 (20) 5 (16) Amiens 4 (14) 5 (16) http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 4 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al Table 3. Demographic characteristics of residents who answered the final evaluation. Characteristic Control group (n=22), n (%) WhatsApp group (n=22), n (%) Year of residency First 9 (41) 9 (41) Second 13 (59) 13 (59) Sex Male 14 (64) 16 (73) Female 8 (36) 6 (27) University hospital Rouen 10 (45) 10 (45) Lille 6 (27) 5 (23) Caen 5 (23) 4 (18) Amiens 1 (5) 2 (10) Table 4. Demographic characteristics of residents who did not answer the final evaluation. Characteristic Control group (n=8), n (%) WhatsApp group (n=8), n (%) Year of residency First 5 (63) 3 (37) Second 3 (37) 5 (63) Sex Male 3 (37) 5 (63) Female 5 (63) 3 (37) University hospital Rouen 0 (0) 0 (0) Lille 4 (50) 4 (50) Caen 1 (13) 1 (13) Amiens 3 (37) 3 (37) of the perceived quality of educational materials (9/10 Use of WhatsApp and Residents’ Satisfaction [interquartile range 8-10] vs 8/10 [interquartile range 8-10]; The residents of the two groups who filled the final evaluation P=.15), the usefulness and relevance of clinical cases (10/10 were asked to fill an online satisfaction questionnaire. Twenty [interquartile range 8-10] vs 9/10 [interquartile range 7-10]; (67%) residents in the WhatsApp group and 13 (43%) residents P=.40), the quantity of teaching documents used by the residents in the control group answered this questionnaire. All the scores (in the WhatsApp group, 14 residents [70%] used more than from the satisfaction evaluation had a non-normal distribution. 50% of the documents and 6 [30%] used less than 50% of the There was a difference between the WhatsApp and control documents; in the control group, 10 residents [77%] used more groups, with the WhatsApp group showing a better global than 50% of the documents and 3 [23%] used less than 50% of satisfaction rate (8/10 [interquartile range, 8-9] vs 8/10 the documents; P=.66), or the time spent working on the [interquartile range 8-8]; P=.049), a better feeling that the program (in the WhatsApp group, 2 residents [10%] spent lessons met the learning objectives (10/10 [interquartile range between 5 h and 10 h and 18 [90%] spent between 1 h and 5 h; 8-10] vs 8/10 [interquartile range 7-10]; P=.03), and a feeling in the control group, 4 residents [31%] spent between 5 h and that the teachers were more available (10/10 [interquartile range 10 h and 9 [69%] spent between 1 h and 5 h; P=.18). Textbox 9-10] vs 9/10 [interquartile range 8-10]; P=.007). We found no 1 presents quotes from the free comments section of the differences between the WhatsApp and control groups in terms satisfaction questionnaire of the WhatsApp group. http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 5 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al Figure 1. Flowchart of the study. e-learning: electronic learning; MCQ: multiple-choice question; SCT: script concordance test. Textbox 1. Quotes from the free comments section of the satisfaction questionnaire (WhatsApp group residents). “For participation to the Friday clinical case it depends on the availability of everyone. The fact that it is on whatsapp makes it easier to communicate and ask questions. Having notifications is more motivating to consult documents than on a platform.” “Having what's app notifications allows me to be more assiduous, the possibility to ask questions directly in the conversation is a big advantage, it sometimes allows small discussions, so very useful. Great classes, interesting cases, and not feeling evaluated is fun. Suggestion: a new session.” “The documents were very well done, difficult on Friday to answer all the questions of the clinical cases online according to our occupations in the ward.” “Very good idea to teach via WhatsApp, which allows to be informed quickly of the presence of new educational documents and to have regular reminder shots since the notifications are displayed. Doing clinical cases on the application during a day with the participation of several people is very instructive. The only problem is that the documents are difficult to consult on a small telephone, perhaps it would be necessary to adapt the documents in the form of slides format telephone. Otherwise it was great! High quality educational documents. Thank you!” “Interesting to be able to consult documents via whatsapp. As far as Friday clinical cases are concerned, it is quite difficult to switch between ward presence or other obligations and whatsapp.” “Very nice project. I think it's useful to have cards per whatsapp but the flow was too high: 4 documents per day, we end up having too much delay in the readings.” “The idea is good but it's quite laborious to read lessons on a mobile phone, especially long pdf. The well ventilated and clear synthetic dcouments [sic] are on the other hand interesting. It is also interesting to be able to ask questions directly and get quick answers. But it can't replace classical education.” http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 6 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al satisfaction for otorhinolaryngology and head and neck surgery Discussion disorders in undergraduate medical students [28]. Despite the absence of differences in knowledge gain in the mobile Principal Results interactive multimedia group, satisfaction was higher in the This randomized, multicenter study is the first to focus on the mobile group (like in our cohort). Therefore, we can assume impact of WhatsApp on clinical reasoning in medical students. that our data are concordant with the literature on m-learning. We found that the use of WhatsApp, instead of a traditional Finally, the daily use of WhatsApp for medical education e-learning platform, to teach a specific topic was associated probably contributes to the dispersion of attention linked to the with worse clinical reasoning despite better global appreciation. use of the smartphone. In view of these results, it does not seem Comparison With Prior Work justified to continue to develop WhatsApp for teaching medical Several recent studies have reported the potential interest of reasoning to medical residents. However, the targeted use of specific smartphone apps in medical education, but our objective WhatsApp with other educational objectives (eg, medical was to assess the interest of a very widely used nonmedical app imaging or video) remains to be evaluated and should be the (thus easily usable by all) for teaching [27-29]. Given that subject of future randomized studies. It is known that blended WhatsApp allows interaction between teachers and students, learning can have a beneficial effect on knowledge acquisition with the possibility of discussing clinical cases, we believed its in health professions [30]. Thus, it might also be interesting to use would improve medical reasoning, as previously described study the use of WhatsApp as a complement to another form for face-to-face practice exchange groups [12]. We did not find of teaching. Given the increasing use of smartphones by health any difference in the global amount of work or the number of workers, it also seems appropriate to consider future work to educational documents consulted, which is consistent with assess the quality of clinical reasoning between two populations similar personal work between the two groups. It has been of physicians with or without usual smartphone use in hospitals. shown that e-learning methods improve the medical knowledge Limitations of health care professionals [28]. The absence of a difference Our study has several limitations. First, the Cronbach coefficient in the medical knowledge assessed by MCQs shows that the alpha in our SCT evaluation was low. The minimum coefficient weakness of clinical reasoning related to WhatsApp is not usually retained for normative evaluations is 0.7, but in our related to less knowledge of the subject. We can therefore work the evaluation was only formative and integrated into assume that this decrease in the quality of reasoning is directly teaching. The limited number of SCTs probably explains this related to WhatsApp or the use of a smartphone. It is likely that low coefficient. However, teaching in a specific and specialized reading on WhatsApp between two other activities was less area made it difficult to find at least 60 SCTs (as is usually effective than time spent solely on an e-learning platform. A recommended) without redundancy [18]. Second, residents’ recent study showed that a smartphone app dedicated to teaching participation in our work was limited: Only 62 of 142 residents medical students Dermatology, in combination with traditional participated. As previously observed, self-training with teaching, improved medical knowledge measured by MCQs e-learning is impacted by a significant dropout rate [26]. In our [27]. Although we did not find any improvement in medical work, only 22 of the 30 residents participated in the final knowledge in our work, the smartphone was seen as an evaluation. As this teaching was optional, participation in our alternative to conventional e-learning and not as a complement. study represented additional personal work for the residents. It It is interesting to note that in the literature, most of the is therefore possible that the majority of residents were educational benefits reported with smartphone use stem from discouraged by this prospect. In addition, residents without very “visual” specialties (Dermatology or Pathology) and that smartphones or those who did not wish to use WhatsApp this tool, which allows easy communication of iconography, is logically refused to participate. Third, we could not prevent probably more relevant in this context than in “less visual” cross-communication among students while they answered the medical specialties [10,27]. SCTs and MCQs, and the residents could have communicated Residents pointed out two limitations: the difficulty of with each other during the final evaluation. The fact that this participating in clinical cases on Friday in parallel with their evaluation was not sanctioned and had no value, as it was not usual activities and the difficulty in referring to documents on integrated into usual teaching methods, probably limited this small smartphone screens. Unlike for practice exchange groups, communication. Finally, we did not use a prevalidated there was no time dedicated specifically to clinical case questionnaire to measure satisfaction. As we wanted to evaluate resolution on WhatsApp, and residents had to respond in specific points related to the use of WhatsApp in our population, addition to their usual activities [12]. This probably favored a we created a new dedicated questionnaire, but this choice made multitasking activity with a difficulty to focus on the it more difficult to compare our satisfaction results to those of pedagogical content. However, it is interesting to note that the others. comments from WhatsApp residents were very positive, with Conclusions a higher overall satisfaction rating. The novelty and originality of the concept probably contributed to this satisfaction, but it Compared to traditional e-learning, the use of WhatsApp as an underlines the fact that the students were not aware of the m-learning method for residents teaching is associated with possible negative impact of the use of WhatsApp. A recent worse clinical reasoning despite better global appreciation. The randomized pedagogic study assessed the impact of learning use of the WhatsApp app probably contributes to the dispersion modules using m-learning on knowledge gain, skill gain, and of attention linked to the use of the smartphone. http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 7 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al Acknowledgments The authors are grateful to Nikki Sabourin-Gibbs, Rouen University Hospital, for her help in editing the manuscript; to all the senior anesthesiologists from Caen and Rouen University Hospitals who agreed to be experts for SCT answers; and to all the anesthesiology residents who agreed to participate in this work. Funding support was provided solely from departmental sources. Conflicts of Interest None declared. Multimedia Appendix 1 Teaching documents especially prepared for easy readability on a smartphone (in French with English translation). [PDF File (Adobe PDF File), 3MB-Multimedia Appendix 1] Multimedia Appendix 2 "Step-by-step" clinical cases (in French with English translation). [PDF File (Adobe PDF File), 428KB-Multimedia Appendix 2] Multimedia Appendix 3 Several examples of the use of WhatsApp for learning purposes during the protocol (screenshots in French with English translation). [PDF File (Adobe PDF File), 741KB-Multimedia Appendix 3] Multimedia Appendix 4 Script concordance test used for the final evaluation (in French with English translation). [PDF File (Adobe PDF File), 632KB-Multimedia Appendix 4] Multimedia Appendix 5 Multiple-choice questions used for the final evaluation (in French with English translation). [PDF File (Adobe PDF File), 720KB-Multimedia Appendix 5] Multimedia Appendix 6 Online satisfaction questionnaire (in French with English translation). [PDF File (Adobe PDF File), 550KB-Multimedia Appendix 6] Multimedia Appendix 7 CONSORT‐EHEALTH checklist (V 1.6.1). [PDF File (Adobe PDF File), 2MB-Multimedia Appendix 7] References 1. Ellaway R, Masters K. AMEE Guide 32: e-Learning in medical education Part 1: Learning, teaching and assessment. Med Teach 2008 Jun;30(5):455-473. [doi: 10.1080/01421590802108331] [Medline: 18576185] 2. Alsobayel H. Use of Social Media for Professional Development by Health Care Professionals: A Cross-Sectional Web-Based Survey. JMIR Med Educ 2016 Sep 12;2(2):e15 [FREE Full text] [doi: 10.2196/mededu.6232] [Medline: 27731855] 3. Chan WS, Leung AY. Use of Social Network Sites for Communication Among Health Professionals: Systematic Review. J Med Internet Res 2018 Mar 28;20(3):e117. [doi: 10.2196/jmir.8382] [Medline: 29592845] 4. Christensen MA, Bettencourt L, Kaye L, Moturu ST, Nguyen KT, Olgin JE, et al. Direct Measurements of Smartphone Screen-Time: Relationships with Demographics and Sleep. PLoS One 2016;11(11):e0165331 [FREE Full text] [doi: 10.1371/journal.pone.0165331] [Medline: 27829040] 5. Eggermont S, Bloemendaal PM, van Baalen JM. E-learning any time any place anywhere on mobile devices. Perspect Med Educ 2013 Mar 30 [FREE Full text] [doi: 10.1007/s40037-013-0045-4] [Medline: 23670682] http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 8 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al 6. Masters K, Ellaway RH, Topps D, Archibald D, Hogue RJ. Mobile technologies in medical education: AMEE Guide No. 105. Med Teach 2016 Jun;38(6):537-549. [doi: 10.3109/0142159X.2016.1141190] [Medline: 27010681] 7. Meola A. Business Insider French. WhatsApp is the most popular chat app in more than half the world Internet URL: http:/ /www.businessinsider.fr/us/whatsapp-is-the-most-popular-chat-app-in-more-than-half-the-world-2016-5/ [accessed 2018-11-12] [WebCite Cache ID 73sJKGsIb] 8. AlFaris E, Irfan F, Ponnamperuma G, Jamal A, Van der Vleuten C, Al Maflehi N, et al. The pattern of social media use and its association with academic performance among medical students. Med Teach 2018 Dec;40(sup1):S77-S82. [doi: 10.1080/0142159X.2018.1465536] [Medline: 29732945] 9. Jamal A, Temsah M, Khan SA, Al-Eyadhy A, Koppel C, Chiang MF. Mobile Phone Use Among Medical Residents: A Cross-Sectional Multicenter Survey in Saudi Arabia. JMIR Mhealth Uhealth 2016 May 19;4(2):e61 [FREE Full text] [doi: 10.2196/mhealth.4904] [Medline: 27197618] 10. Goyal A, Tanveer N, Sharma P. WhatsApp for Teaching Pathology Postgraduates: A Pilot Study. J Pathol Inform 2017;8:6 [FREE Full text] [doi: 10.4103/2153-3539.201111] [Medline: 28400995] 11. Raiman L, Antbring R, Mahmood A. WhatsApp messenger as a tool to supplement medical education for medical students on clinical attachment. BMC Med Educ 2017 Jan 06;17(1):7 [FREE Full text] [doi: 10.1186/s12909-017-0855-x] [Medline: 28061777] 12. Compère V, Abily J, Moriceau J, Gouin A, Veber B, Dupont H, et al. Residents in tutored practice exchange groups have better medical reasoning as measured by script concordance test: a controlled, nonrandomized study. J Clin Anesth 2016 Aug;32:236-241. [doi: 10.1016/j.jclinane.2016.03.012] [Medline: 27290981] 13. Ducos G, Lejus C, Sztark F, Nathan N, Fourcade O, Tack I, et al. The Script Concordance Test in anesthesiology: Validation of a new tool for assessing clinical reasoning. Anaesth Crit Care Pain Med 2015 Feb;34(1):11-15. [doi: 10.1016/j.accpm.2014.11.001] [Medline: 25829309] 14. Bursztejn A, Cuny J, Adam J, Sido L, Schmutz J, de Korwin J, et al. Usefulness of the script concordance test in dermatology. J Eur Acad Dermatol Venereol 2011 Dec;25(12):1471-1475. [doi: 10.1111/j.1468-3083.2011.04008.x] [Medline: 22077872] 15. Dutton RP, Stansbury LG, Leone S, Kramer E, Hess JR, Scalea TM. Trauma mortality in mature trauma systems: are we doing better? An analysis of trauma mortality patterns, 1997-2008. J Trauma 2010 Sep;69(3):620-626. [doi: 10.1097/TA.0b013e3181bbfe2a] [Medline: 20093983] 16. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA 2008 Sep 10;300(10):1181-1196. [doi: 10.1001/jama.300.10.1181] [Medline: 18780847] 17. Kulier R, Coppus SFPJ, Zamora J, Hadley J, Malick S, Das K, et al. The effectiveness of a clinically integrated e-learning course in evidence-based medicine: a cluster randomised controlled trial. BMC Med Educ 2009 May 12;9:21 [FREE Full text] [doi: 10.1186/1472-6920-9-21] [Medline: 19435520] 18. Lubarsky S, Dory V, Duggan P, Gagnon R, Charlin B. Script concordance testing: from theory to practice: AMEE guide no. 75. Med Teach 2013;35(3):184-193. [doi: 10.3109/0142159X.2013.760036] [Medline: 23360487] 19. Dcode. Tirage au Sort URL: https://www.dcode.fr/tirage-au-sort-nombre-aleatoire [accessed 2018-11-14] [WebCite Cache ID 73vLExugV] 20. Rossaint R, Bouillon B, Cerny V, Coats TJ, Duranteau J, Fernández-Mondéjar E, et al. The European guideline on management of major bleeding and coagulopathy following trauma: fourth edition. Crit Care 2016 Apr 12;20:100 [FREE Full text] [doi: 10.1186/s13054-016-1265-x] [Medline: 27072503] 21. Duranteau J, Asehnoune K, Pierre S, Ozier Y, Leone M, Lefrant J. Recommandations sur la réanimation du choc hémorragique. Anesthésie & Réanimation 2015 Feb;1(1):62-74. [doi: 10.1016/j.anrea.2014.12.007] 22. Kassirer JP. Teaching clinical reasoning: case-based and coached. Acad Med 2010 Jul;85(7):1118-1124. [Medline: 20603909] 23. Schmidt HG, Mamede S. How to improve the teaching of clinical reasoning: a narrative review and a proposal. Med Educ 2015 Oct;49(10):961-973. [doi: 10.1111/medu.12775] [Medline: 26383068] 24. Enser M, Moriceau J, Abily J, Damm C, Occhiali E, Besnier E, et al. Background noise lowers the performance of anaesthesiology residents' clinical reasoning when measured by script concordance: A randomised crossover volunteer study. Eur J Anaesthesiol 2017 Dec;34(7):464-470. [doi: 10.1097/EJA.0000000000000624] [Medline: 28394819] 25. Faculté de Médecine - Université de Montréal Internet. Obtenir les scores de vos TCS par voie informatique - Centre de pédagogie appliquée aux sciences de la santé (CPASS) URL: https://www.cpass.umontreal.ca/recherche/ groupe-de-recherche-cpass/axes-de-recherches/concordance/tcs/corriger_tcs/ [accessed 2018-11-14] [WebCite Cache ID 73vBHLU6Q] 26. Moreira IC, Ventura SR, Ramos I, Rodrigues PP. Development and assessment of an e-learning course on breast imaging for radiographers: a stratified randomized controlled trial. J Med Internet Res 2015;17(1):e3 [FREE Full text] [doi: 10.2196/jmir.3344] [Medline: 25560547] 27. Fransen F, Martens H, Nagtzaam I, Heeneman S. Use of e-learning in clinical clerkships: effects on acquisition of dermatological knowledge and learning processes. Int J Med Educ 2018 Jan 17;9:11-17 [FREE Full text] [doi: 10.5116/ijme.5a47.8ab0] [Medline: 29352748] http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 9 (page number not for citation purposes) XSL FO RenderX JMIR MHEALTH AND UHEALTH Clavier et al 28. Lee L, Chao Y, Huang C, Fang J, Wang S, Chuang C, et al. Cognitive Style and Mobile E-Learning in Emergent Otorhinolaryngology-Head and Neck Surgery Disorders for Millennial Undergraduate Medical Students: Randomized Controlled Trial. J Med Internet Res 2018 Feb 13;20(2):e56 [FREE Full text] [doi: 10.2196/jmir.8987] [Medline: 29439943] 29. Shaw CM, Tan SA. Integration of mobile technology in educational materials improves participation: creation of a novel smartphone application for resident education. J Surg Educ 2015;72(4):670-673. [doi: 10.1016/j.jsurg.2015.01.015] [Medline: 25823746] 30. Liu Q, Peng W, Zhang F, Hu R, Li Y, Yan W. The Effectiveness of Blended Learning in Health Professions: Systematic Review and Meta-Analysis. J Med Internet Res 2016;18(1):e2 [FREE Full text] [doi: 10.2196/jmir.4807] [Medline: 26729058] Abbreviations e-learning: electronic learning MCQ: multiple-choice questions m-learning: mobile learning SCT: script concordance test Edited by G Eysenbach; submitted 15.11.18; peer-reviewed by LA Lee, VK Bandi, A Paglialonga; comments to author 10.01.19; revised version received 17.01.19; accepted 23.02.19; published 09.04.19 Please cite as: Clavier T, Ramen J, Dureuil B, Veber B, Hanouz JL, Dupont H, Lebuffe G, Besnier E, Compere V JMIR Mhealth Uhealth 2019;7(4):e12825 URL: http://mhealth.jmir.org/2019/4/e12825/ doi: 10.2196/12825 PMID: 30964435 ©Thomas Clavier, Julie Ramen, Bertrand Dureuil, Benoit Veber, Jean-Luc Hanouz, Hervé Dupont, Gilles Lebuffe, Emmanuel Besnier, Vincent Compere. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 09.04.2019. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included. http://mhealth.jmir.org/2019/4/e12825/ JMIR Mhealth Uhealth 2019 | vol. 7 | iss. 4 | e12825 | p. 10 (page number not for citation purposes) XSL FO RenderX

Journal

JMIR mHealth and uHealthJMIR Publications

Published: Apr 9, 2019

Keywords: education, medical, graduate/methods; educational measurement; anesthesiology; internship and residency; trauma; hemorrhage; mobile applications; WhatsApp; smartphone; teaching materials; mobile phone

There are no references for this article.