Social media posts and online search behaviour as early-warning system for MRSA outbreaks

Social media posts and online search behaviour as early-warning system for MRSA outbreaks Background: Despite many preventive measures, outbreaks with multi-drug resistant micro-organisms (MDROs) still occur. Moreover, current alert systems from healthcare organizations have shortcomings due to delayed or incomplete notifications, which may amplify the spread of MDROs by introducing infected patients into a new healthcare setting and institutions. Additional sources of information about upcoming and current outbreaks, may help to prevent further spread of MDROs. The study objective was to evaluate whether methicillin-resistant Staphylococcus aureus (MRSA) outbreaks could be detected via social media posts or online search behaviour; if so, this might allow earlier detection than the official notifications by healthcare organizations. Methods: We conducted an exploratory study in which we compared information about MRSA outbreaks in the Netherlands derived from two online sources, Coosto for Social Media, and Google Trends for search behaviour, to the mandatory Dutch outbreak notification system (SO-ZI/AMR). The latter provides information on MDRO outbreaks including the date of the outbreak, micro-organism involved, the region/location, and the type of health care organization. Results: During the research period of 15 months (455 days), 49 notifications of outbreaks were recorded in SO-ZI/ AMR. For Coosto, the number of unique potential outbreaks was 37 and for Google Trends 24. The use of social media and online search behaviour missed many of the hospital outbreaks that were reported to SO-ZI/AMR, but detected additional outbreaks in long-term care facilities. Conclusions: Despite several limitations, using information from social media and online search behaviour allows rapid identification of potential MRSA outbreaks, especially in healthcare settings with a low notification compliance. When combined in an automated system with real-time updates, this approach might increase early discovery and subsequent implementation of preventive measures. Keywords: Methicillin-resistant Staphylococcus aureus, MRSA, Social media monitoring, Outbreaks, Google trends, Nowcasting * Correspondence: tom.vandebelt@radboudumc.nl Radboud REshape Innovation Center, Radboudumc University Medical Center, Geert Grooteplein Zuid 10, 6525, GA, Nijmegen, Netherlands Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 2 of 10 Background engines) could be a valuable source of information about The Dutch healthcare system applies strict infection potential MDRO outbreaks. Interestingly, online search control guidelines regarding multi-drug resistant behaviour has already successfully been used to detect micro-organisms (MDROs), including the “Search & influenza outbreaks based on search entries [5], and dis- Destroy” guideline for methicillin-resistant Staphylococcus ease outbreaks in general [6]. Moreover, social media aureus (MRSA), which was extended to other MDROs in have been used to monitor the quality of healthcare in- 2011 [1, 2]. Despite the implementation of these guide- stitutions on, for example, hygiene and expertise [7]. lines, outbreaks with MDROs still occur. Reasons may be The aim of the current study was to evaluate whether a temporary lack of compliance with existing guidelines, Dutch MRSA outbreaks could be detected via social human error, or spread from infected patients not falling media posts or online search behaviour; and if so, into a high-risk category that would warrant screening whether these data sources might allow earlier detection and isolation on admission. One of the defined than the official notification to SO-ZI/AMR by the hos- high-risk-categories of the Dutch MRSA/MDRO guide- pital. In addition, as reporting outbreaks in nursing line, are patients originating from a healthcare setting with homes is still voluntary, we also evaluated whether an ongoing MRSA/MDRO outbreak. In the past, hospitals screening of social media posts and/or online search be- were supposed to inform each other about outbreaks and haviour would help to identify outbreaks in these health- possible colonized or infected patients they exchange, but care institutions. In this study, we focused on MRSA. the report itself as well as the way of communication were non-standardized and voluntarily. As of 2012, all hospitals Methods report their MDRO outbreaks to a central point (SO-ZI/ Design and Setting AMR), which was initiated and established by the Dutch We conducted an exploratory study in which we com- Society of Clinical Microbiology (NVMM) after the first pared information about MRSA outbreaks derived from carbapenem-resistant Enterobacteriaceae (CRE)-outbreak social media and online search behaviour to the official in the Netherlands [3]. SO-ZI/AMR contains a database Dutch reference standard. MRSA specific searches were with information about the outbreak such as date and performed for the time period between January 1st, 2015 duration of the outbreak, organization name affected loca- and March 31st, 2017. As reference standard for MRSA tion(s) and the micro-organism in question. Outbreaks outbreaks, SO-ZI/AMR was used [8]. It provides infor- that need to be reported to SO-ZI/AMR are defined as: mation on official outbreaks including the date of the Outbreaks which influence, or have the potential to nega- outbreak and the region and type of health care facility tively influence, access to care, such as in case of (possible) (e.g. hospital or nursing home). The geographical scope closure of a department or part of it, and/or outbreaks of the study was The Netherlands; therefore, we only with continuous transmission despite (infection) control searched using the Dutch language. measures [4]. When reporting outbreaks became part of the profes- sional guidelines, it became essentially mandatory. How- Social media sources and online search behaviour ever, reporting outbreaks is currently only mandatory for To capture social media posts about MRSA (all publicly hospitals, and not for nursing homes or other health shared social media posts by individuals or organiza- care institutions. Once reported, an outbreak is, with a tions), we used Coosto, a social media monitoring tool short delay, immediately visible for all users. The task of [9]. This tool has proven to be a valuable source of social SO-ZI/AMR is not only to collect reports and report media information and is currently in use by the Dutch outbreaks to professionals, but also to monitor the de- government to monitor the quality of healthcare organi- velopment of the outbreak, and, if needed, to support zations [7]. It provides the exact time and, if available, the control efforts. Still, the alert messages from some the location of the message in various social media hospitals seem to come late or not at all, with the risk of sources, including Facebook and Twitter. Presently, its introducing patients infected with an MDRO into a new database includes posts in the Dutch language. healthcare setting without warning and increasing the In addition, Google Trends was used to assess online probability of spreading the outbreak. Therefore, there is search behaviour [10]. This tool provides insight into the a need for additional sources of information about search behaviour based on specific searches performed current and potentially upcoming outbreaks, to increase in Google Search. It provides the relative frequency of outbreak preparedness. searches for different countries and regions. Google Since an increasing number of people use social Trends has been used for early detection of influenza media, such as Facebook and Twitter, to share informa- outbreaks [6]. Although multiple search engines are be- tion and the Internet as source for news, social media ing used in The Netherlands, we limited our searches to posts and online search behaviour (e.g., via search Google Trends since Google covers over 80% of Internet van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 3 of 10 searches, capturing the overall majority of Internet outbreak occurred at least once every 3 months, we used searches [11]. the mean search interest of 3 months in our analyses. We extracted search interest per day, as well as geo- Data extraction graphical information (province) for days with potential We searched Coosto for publicly available social media outbreaks. posts about MRSA with the following search query: (“mrsa” OR “methicillin-resistant Staphylococcus aureus” Statistical analysis OR “methicillin resistant Staphylococcus aureus” OR All statistical analyses were done in SPSS version 22. “meticilline-resistente Staphylococcus aureus” OR “meti- The database with search results from Coosto and Goo- cilline resistente Staphylococcus aureus”). Several prelim- gle Trends was compared to data from the official inary searches revealed that both Dutch and English SO-ZI/AMR database. In case of consecutive days with names had to be included, since English terminology the same potential outbreak, we defined this as a single was sometimes used in Dutch social media posts and the outbreak, both for Coosto and Google Trends. To assess combination maximized the number of hits. Further- the validity of Coosto to detect potential MRSA out- more, the word ‘outbreak’ was not included in the search breaks, we calculated the overall sensitivity, specificity, query, since the preliminary searches showed that positive predictive value (PPV), and negative predictive searching for ‘outbreak’ resulted in a large number of ir- value (NPV) with 95% confidence intervals (CIs). Fur- relevant hits, and that combining ‘outbreak’ with MRSA thermore, we stratified the analyses by type of healthcare via Boolean search (“AND”) negatively affected the sen- institution affected. sitivity of the search. Based on the results of the prelim- For the Google Trends data, a search score was calcu- inary searches, we performed manual inspections of lated for each day: results with 25, 15 and 10 hits per day. In general, the Google Trends Score = (relative search volume - mean lowest number of hits per day would result the highest relative search volume) / standard deviation. number of potential outbreaks. However, from the above with the mean relative search volume and standard de- comparison, we concluded found that 10 posts per day viation based on the preceding 3 months. Using SO-ZI/ was the minimum number of hits, or ‘critical mass’ [7] AMR as the reference standard, we calculated the area to identify a potential MRSA outbreak, Consequently, under the curve (AUC) for 7 days before until 7 days we set (≥10 hits) as criterion for a potential outbreak. after an outbreak. Furthermore, as the optimal cut-off For all posts on a specific day, that met this criterion, we value to detect an outbreak based on online search be- identified whether a potential outbreak was discussed, haviour is unknown, we determined the Google Trends meaning that MRSA was mentioned in relation to a Score that maximized the sum of sensitivity and specifi- Dutch healthcare institution and indicating a present or city. For the present study, we used a cut-off value of potential outbreak. The latter could consist of (but not 2*SD to detect a potential outbreak with Google Trends. limited to) patients, relatives or employees found or sus- Finally, we calculated the Pearson correlation coefficient pected with MRSA, hospital wards closed due to MRSA, any other information about a present or expected Table 1 Characteristics of MRSA outbreaks detected in SO-ZI/ MRSA outbreak shared by the institution, its employees, AMR and Coosto government, or other any other individual (e.g., patients SO-ZI/AMR Coosto Google Trends or relatives). Days meeting all criteria were marked as Number of days with ≥10 posts 297 ‘representing a potential MRSA outbreak’ and all other Excluded 260 days as ‘not representing a potential MRSA outbreak’. Multiple days 22 Dates, number of hits, institution and geographical area and outbreak (YES/NO) were subsequently stored in a New research 33 research database. Unrelated messages 59 Regarding the searches in Google Trends, we used General info concerning MRSA 115 similar terms, but with individual searches for each term. Articles about hygiene and meat 31 Google Trends presents search interest of topics on a Total number of outbreaks 49 37 24 scale from 0 to 100 per day instead of the absolute num- University hospital 8 1 ber of searches, thus every search will have at least one day with the maximum score of 100, even when absolute General hospital 22 17 search numbers are low during a particular time period Nursing home 7 16 (e.g., a time period without any outbreaks). This system Other 3 3 characteristic required a different way of defining days Unknown 9 0 with potential outbreaks. Assuming that a Dutch MRSA van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 4 of 10 Fig. 1 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations to assess the association between the number of posts social media posts, of which 1 referred to an academic related to MRSA detected with Coosto and the Google hospital, 17 to a general hospital, 16 to a nursing home, Trends Score. and 3 to other types of institutions. Google Trends re- sulted in 24 potential outbreaks. Results Figures 1, 2, 3, 4, 5, 6, 7, 8, and 9 show the information During the research period of 15 months (455 days), 49 on MRSA outbreaks originating from the three data outbreaks were reported to SO-ZI/AMR (Table 1). Using sources in each quarter of a year. In only 4 outbreaks Coosto, 37 potential outbreaks were detected based on did all three sources show a (potential) outbreak, with in Fig. 2 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 5 of 10 Fig. 3 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations general the online sources detecting the outbreak 1– by Google Trends. Correlation between the number of 2 days before the official notification date in SO-ZI/ posts related to MRSA detected with Coosto and the AMR. In 48 cases, Coosto and/or Google Trends indi- Google Trends Score was 0.42, p < 0.001). cated a (potential) outbreak without notification in Validity comparisons for Coosto-detected MRSA out- SO-ZI/AMR, whereas in 41 cases, MRSA outbreaks breaks showed an overall sensitivity of 0.20 (95% CI 0.10– were notified to SO-ZI/AMR that went unnoticed by the 0.34) and an overall specificity of 0.96 (95% CI 0.95–0.98), online data sources. In 4 cases, a (potential) outbreak whereas the PPV and NPV were 0.27 (95% CI 0.16–0.42) was detected by both SO-ZI-AMR and Coosto, and not and 0.95 (95% CI 0.92–0.96), respectively (Table 2). After Fig. 4 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 6 of 10 Fig. 5 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations stratification for type of healthcare institution, sensitivity 0.63 (95% CI 0.54–0.73) for MRSA outbreaks in hospi- ranged between 0 for other and unknown locations to tals. With the optimal cut-off for the Google Trends 0.23 (95% CI 0.08–0.45) for general hospitals. Specificity Score, sensitivity was higher for any outbreak compared was ≥0.98 for all types of institutions. with hospital outbreaks only (0.90 vs. 0.43), whereas spe- The validity comparisons for Google Trends to detect cificity was higher for hospital outbreaks (0.28 vs. 0.79). MRSA outbreaks compared with SO-ZI/AMR as the ref- The AUC based on the Google Trends Score 1 day be- erence standard are shown in Table 3. On the exact date fore the official notification was similar to the AUC on the outbreak was notified to SO-ZI/AMR, the AUC was the day of notification. On the other days relative to the 0.59 (95% CI 0.51–0.67) for any MRSA outbreak and notification of the outbreaks, the AUC was decreased. Fig. 6 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/ AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 7 of 10 Fig. 7 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/ AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations Discussion These promising findings suggest that supervisory bod- Principal findings ies such as SO-ZI/AMR may enrich their palette of data In this study, we compared information about potential sources with more dynamic information from social MRSA outbreaks retrieved from social media posts and media and other online sources such as search engine online search behaviour in The Netherlands to the na- data. However, the validity of the online sources Coosto tional notification reference standard. We found that and Google trends needs further investigation. Some simple online (social media) searches do provide add- things need to be discussed. itional information about potential MRSA outbreaks in The sensitivity of the social media monitoring tool The Netherlands compared to the reference standard. Coosto to detect MRSA outbreaks was low and therefore a Fig. 8 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 8 of 10 Fig. 9 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations substantial number of true outbreaks will be missed when more reluctant to post a message about the outbreak on so- relying on this data source. However, its specificity was cial media than to search for information online. In high, indicating a relatively small number of false positive addition, it is impossible to distinguish between online outbreaks detected by Coosto. Interestingly, the opposite searches for actual outbreaks and searches for random is- was observed for Google Trends, with a higher sensitivity sues related to MRSA using Google Trends. and lower specificity, indicating that Google Trends detects To the best of our knowledge, this was the first study more potential MRSA outbreaks but that many of these using online search engines for social media posts and represent false positive signals. This difference between the internet search behaviour on MRSA outbreak detection. two online data sources may be explained by their nature: A study by Lui et al. used search terms from the social social media posts provide more detailed information on media platform Baidu to identify Noro virus epidemics MRSA outbreaks than online searches, but the patients and [12]. They concluded that several limitations exist to healthcare workers involved in MRSA outbreaks may be using Internet to monitor epidemics but that it still Table 2 Validity comparisons for Coosto-detected MRSA outbreaks with SO-ZI/AMR as the reference standard MRSA outbreaks Number of days Validity TP FP FN TN Se (95% CI) Sp (95% CI) PPV (95% CI) NPV (95% CI) All 10 27 39 722 20 (10–34) 96 (95–98) 27 (16–42) 95 (92–96) Any hospital 5 13 25 755 17 (6–35) 98 (97–99) 28 (13–50) 97 (96–97) University hospital 0 1 8 791 0 100 (99–100) 0 99 (99–99) General hospital 5 12 17 764 23 (8–45) 98 (97–99) 29 (14–52) 98 (97–98) Any hospital and unknown location 6 12 33 747 15 (6–31) 98 (97–99) 33 (17–56) 96 (95–96) University hospital and unknown location 0 1 17 783 0 100 (99–100) 0 98 (98–98) General hospital and unknown location 6 11 25 756 19 (7–37) 99 (97–99) 35 (18–58) 97 (96–97) Nursing home 1 15 6 778 13 (0–53) 98 (97–99) 6 (1–31) 99 (99–99) Nursing home and unknown location 2 14 14 770 13 (2–38) 98 (97–99) 13 (3–37) 98 (98–99) Other location 0 3 3 795 0 100 (99–100) 0 100 (100–100) Other location and unknown location 0 3 12 786 0 100 (99–100) 0 99 (98–99) CI confidence interval, FN false negative, FP false positive, NPV negative predictive value, PPV positive predictive value, Se sensitivity, Sp specificity, TN true negative, TP true positive van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 9 of 10 Table 3 Validity comparisons for Google Trends to detect MRSA outbreaks with SO-ZI/AMR as the reference standard. Day 0 indicates the day the outbreak was reported to SO-ZI/AMR Day Any MRSA outbreak MRSA outbreak in hospital a a AUC (95% CI) Optimal cutoff Se Sp AUC (95% CI) Optimal cutoff Se Sp −7 0.61 (0.54–0.69) −0.2018 75 47 0.59 (0.51–0.67) − 0.6507 100 23 −6 0.51 (0.44–0.59) − 0.1208 63 51 0.51 (0.42–0.59) −0.6965 93 21 −5 0.46 (0.38–0.55) −0.4318 71 33 0.48 (0.39–0.58) −0.8801 97 13 −4 0.50 (0.42–0.58) −0.5524 81 28 0.53 (0.44–0.62) −0.5524 87 28 −3 0.46 (0.38–0.54) −0.5377 81 29 0.48 (0.39–0.58) −0.5377 87 29 −2 0.52 (0.44–0.61) 0.1445 42 65 0.47 (0.37–0.57) −0.6828 83 21 −1 0.59 (0.50–0.67) 0.0425 60 61 0.59 (0.47–0.71) 0.0672 67 62 0 0.59 (0.51–0.67) −0.5524 90 28 0.63 (0.54–0.73) 0.3316 43 79 + 1 0.54 (0.46–0.63) −0.1769 63 48 0.49 (0.39–0.59) −1.1493 100 7 + 2 0.49 (0.41–0.57) − 1.1833 100 6 0.46 (0.36–0.55) −0.9484 97 11 +3 0.50 (0.43–0.58) −0.5149 79 29 0.56 (0.46–0.66) −0.1118 63 51 + 4 0.49 (0.41–0.57) −0.6828 90 21 0.49 (0.40–0.59) −0.6828 93 21 + 5 0.44 (0.37–0.51) 0.1674 15 65 0.39 (0.31–0.47) −1.3370 100 4 + 6 0.49 (0.42–0.57) − 1.0128 98 9 0.47 (0.38–0.57) −1.0128 97 9 + 7 0.53 (0.45–0.60) −0.4993 85 30 0.50 (0.40–0.61) − 0.5745 83 27 AUC area under the curve, CI confidence interval, Se sensitivity, Sp specificity Maximizes sensitivity+specificity. Value represents k might have value as additional tool, particularly when outbreaks to specific organizations, since it does not other monitoring systems are lacking. Also, the importance provide specific names or locations. A refinement of of social media as an early warning system in addition to search mechanism of the freely available default Google traditional slow reporting mechanisms has been empha- Trends software might allow an increase of its sensitivity sized [13]. In general, this might be the case for all out- and specificity. breaks as notification is only done after firm confirmation The extent of information patients and caregivers get of the outbreak. Consequently, the notification date in when confronted with an outbreak may influence their SO-ZI/AMR is “delayed” by several days. search behaviour. If the information is complete and of- Additional potential outbreaks were found via social fered right away, as it is customary in many Dutch hospi- media, which were not in the SO-ZI/AMR database. Most tals, it may become more difficult to detect the outbreak of these outbreaks occurred in nursing homes for which no- using social media and search engine behaviour. tification is not mandatory, but on occasion even hospitals were shown as non-compliant in reporting outbreaks. The Conclusions fact that more dynamic data sources could have value com- Despite several limitations including limited validity, pared to traditional slow reporting mechanisms has also using information from social media and online search been recognized in public health, where social media are behaviour results to detect MRSA outbreaks could be an used as an early warning system for disease outbreaks [14]. additional source of information for supervising bodies, particularly when combined in an automated system Strengths and limitations with real-time updates. The main strength of this study is the use of dynamic Funding online content from social media and search engine be- This research was supported by the EPI-Net COMBACTE-MAGNET project. We haviour in combination with an official reference thank the Innovative Medicines Initiative Joint Undertaking for supporting (SO-ZI/AMR). Using predefined selection criteria, this the EPI-Net COMBACTE-MAGNET project (grant agreement number 115737), resources of which include financial contribution from the European Union allowed us to efficiently study the value of social media Seventh Framework Programme (FP7/2007–2013) and European Federation and online search behaviour via both Coosto and Google of Pharmaceutical Industries and Associations companies in-kind contribution. Trends. Coosto on occasion is limited by the fact that it The funder had no role in the study design, data collection, analysis, interpretation of data or writing the manuscript. may not always be possible to determine whether a po- tential outbreak is actually a true outbreak. Google Availability of data and materials Trends has even more difficulty in this determination, The dataset used during the current study are available from the corresponding for using this data source, it is hard to link potential author on reasonable request. van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 10 of 10 Authors’ contributions 12. Liu K, Huang S, Miao ZP, Chen B, Jiang T, Cai G, Jiang Z, Chen Y, Wang Z, TvdB, MvG and AV designed the study, with input from LE and JL and RS. Gu H, Chai C. Jiang J identifying potential norovirus epidemics in China via TvdB, PvS, KS and MvG collected and analyzed data. TvdB produced a draft internet surveillance. J Med Internet Res. 2017;19(8):e282. of the manuscript, and LE, JL, RS, JRB, ET, KS, MvG and AV reviewed it at 13. Denecke K, Dolog P, Smrz P. Making use of social media data in public various stages to its final version. All authors read and approved the final health. In: Proceedings of the 21st international conference on world wide manuscript. web. Mountain View: ACM. p. 243–6. 14. Denecke K. Health web science, health information science, chapter 11, social media for health monitoring p 101–108. Switzerland: Springer International Ethics approval and consent to participate Publishing; 2015. https://doi.org/10.1007/978-3-319-20582-3_11. Since the anonymous data used in this study were derived from the public social media domain without patient involvement, no medical ethical review was needed in the Netherlands. SO-ZI/AMR allowed us to use anonymous (not linked to specific hospital organizations) information from their database. Competing interests AV is Editor-in-Chief of ARIC. All other authors declare that they have no competing interests. Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Author details Radboud REshape Innovation Center, Radboudumc University Medical Center, Geert Grooteplein Zuid 10, 6525, GA, Nijmegen, Netherlands. 2 3 VetEffecT, Bilthoven, The Netherlands. Unidad Clínica de Enfermedades Infecciosas y Microbiología Instituto de Biomedicina de Sevilla (IBiS) /Hospital Universitario Virgen Macarena / CSIC / Departamento de Medicina, Universidad de Sevilla, Sevilla, Spain. Division of Infectious Diseases, Tübingen University Hospital, DZIF Center, Tübingen, Germany. Infectious Diseases, University of Verona, Verona, Italy. Department of Medical Microbiology, Radboudumc, Nijmegen, The Netherlands. Department for Health Evidence, Radboud Institute for Health Sciences, Radboudumc, Nijmegen, The Netherlands. Department of Clinical Microbiology and Infectious Diseases, Canisius-Wilhelmina Hospital, Nijmegen, The Netherlands. Received: 15 January 2018 Accepted: 15 May 2018 References 1. WIP-guideline MRSA. https://www.rivm.nl/dsresource?objectid=3f054354-ff4a- 43ef-91f9-7c6f0417be95.Accessed 30 Aug 2017. 2. WIP-guideline BRMO (Bijzonder Resistente Micro-Organismen). https://www. rivm.nl/dsresource?objectid=b6b99580-44e2-4b9c-8183-52871e61764f&type= org&disposition=inline. Accessed 30 Aug 2017. 3. Van der Bij AK, van Dijk K, Muilwijk J, Thijsen SFT, Notermans DW, de Greeff S. Van de Sande-Bruinsma N. Clinical breakpoint changes and their impact on surveillance of antimicrobial resistance in Escherichia coli causing bacteraemia. Clin Microbiol Infect. 2012;18:E466–74. 4. RNVWS.nl Criteria uitbraak. https://www.nvmm.nl/zorg/uitbraakmeldingen/. Accessed 30 Aug 2017. 5. Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L. Detecting influenza epidemics using search engine query data. Nature. 2009;457:1012–4. 6. Carneiro HA, Mylonakis E. Google trends: a web-based tool for real-time surveillance of disease outbreaks. Clin Infect Dis. 2009;49:1557–64. 7. van de Belt TH, Engelen LJ, Verhoef LM, van der Weide MJ, Schoonhoven L, Kool RB. Using patient experiences on Dutch social media to supervise health care services: exploratory study. J Med Internet Res. 2015;17(1):e7. 8. van der Bij AK, Kardamanidis K, Frakking FN, Bonten MJ. Signaleringsoverleg Ziekenhuisinfecties en Antimicrobiële Resistentie. Nosocomial outbreaks and resistant microorganisms. Ned Tijdschr Geneeskd. 2015;159:A8585. 9. Coosto, Social Media Monitoring, Webcare & Publishing. Coosto B.V. Eindhoven, The Netherlands. Version 2017 Coosto http://www.coosto.nl. Accessed 30 Aug 2017. 10. Google Trends. Google. http://google.com/trends. Accessed 30 Aug 2017. 11. Nationale Search Engine Monitor (2014) (iProspect en RMI) https://www. iprospect.com/nl/nl/onze-blog/nationale-search-engine-monitor-2014/. Accessed 30 Aug 2017. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Antimicrobial Resistance & Infection Control Springer Journals

Loading next page...
 
/lp/springer_journal/social-media-posts-and-online-search-behaviour-as-early-warning-system-mWhktOOZgs
Publisher
Springer Journals
Copyright
Copyright © 2018 by The Author(s).
Subject
Biomedicine; Medical Microbiology; Drug Resistance; Infectious Diseases
eISSN
2047-2994
D.O.I.
10.1186/s13756-018-0359-4
Publisher site
See Article on Publisher Site

Abstract

Background: Despite many preventive measures, outbreaks with multi-drug resistant micro-organisms (MDROs) still occur. Moreover, current alert systems from healthcare organizations have shortcomings due to delayed or incomplete notifications, which may amplify the spread of MDROs by introducing infected patients into a new healthcare setting and institutions. Additional sources of information about upcoming and current outbreaks, may help to prevent further spread of MDROs. The study objective was to evaluate whether methicillin-resistant Staphylococcus aureus (MRSA) outbreaks could be detected via social media posts or online search behaviour; if so, this might allow earlier detection than the official notifications by healthcare organizations. Methods: We conducted an exploratory study in which we compared information about MRSA outbreaks in the Netherlands derived from two online sources, Coosto for Social Media, and Google Trends for search behaviour, to the mandatory Dutch outbreak notification system (SO-ZI/AMR). The latter provides information on MDRO outbreaks including the date of the outbreak, micro-organism involved, the region/location, and the type of health care organization. Results: During the research period of 15 months (455 days), 49 notifications of outbreaks were recorded in SO-ZI/ AMR. For Coosto, the number of unique potential outbreaks was 37 and for Google Trends 24. The use of social media and online search behaviour missed many of the hospital outbreaks that were reported to SO-ZI/AMR, but detected additional outbreaks in long-term care facilities. Conclusions: Despite several limitations, using information from social media and online search behaviour allows rapid identification of potential MRSA outbreaks, especially in healthcare settings with a low notification compliance. When combined in an automated system with real-time updates, this approach might increase early discovery and subsequent implementation of preventive measures. Keywords: Methicillin-resistant Staphylococcus aureus, MRSA, Social media monitoring, Outbreaks, Google trends, Nowcasting * Correspondence: tom.vandebelt@radboudumc.nl Radboud REshape Innovation Center, Radboudumc University Medical Center, Geert Grooteplein Zuid 10, 6525, GA, Nijmegen, Netherlands Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 2 of 10 Background engines) could be a valuable source of information about The Dutch healthcare system applies strict infection potential MDRO outbreaks. Interestingly, online search control guidelines regarding multi-drug resistant behaviour has already successfully been used to detect micro-organisms (MDROs), including the “Search & influenza outbreaks based on search entries [5], and dis- Destroy” guideline for methicillin-resistant Staphylococcus ease outbreaks in general [6]. Moreover, social media aureus (MRSA), which was extended to other MDROs in have been used to monitor the quality of healthcare in- 2011 [1, 2]. Despite the implementation of these guide- stitutions on, for example, hygiene and expertise [7]. lines, outbreaks with MDROs still occur. Reasons may be The aim of the current study was to evaluate whether a temporary lack of compliance with existing guidelines, Dutch MRSA outbreaks could be detected via social human error, or spread from infected patients not falling media posts or online search behaviour; and if so, into a high-risk category that would warrant screening whether these data sources might allow earlier detection and isolation on admission. One of the defined than the official notification to SO-ZI/AMR by the hos- high-risk-categories of the Dutch MRSA/MDRO guide- pital. In addition, as reporting outbreaks in nursing line, are patients originating from a healthcare setting with homes is still voluntary, we also evaluated whether an ongoing MRSA/MDRO outbreak. In the past, hospitals screening of social media posts and/or online search be- were supposed to inform each other about outbreaks and haviour would help to identify outbreaks in these health- possible colonized or infected patients they exchange, but care institutions. In this study, we focused on MRSA. the report itself as well as the way of communication were non-standardized and voluntarily. As of 2012, all hospitals Methods report their MDRO outbreaks to a central point (SO-ZI/ Design and Setting AMR), which was initiated and established by the Dutch We conducted an exploratory study in which we com- Society of Clinical Microbiology (NVMM) after the first pared information about MRSA outbreaks derived from carbapenem-resistant Enterobacteriaceae (CRE)-outbreak social media and online search behaviour to the official in the Netherlands [3]. SO-ZI/AMR contains a database Dutch reference standard. MRSA specific searches were with information about the outbreak such as date and performed for the time period between January 1st, 2015 duration of the outbreak, organization name affected loca- and March 31st, 2017. As reference standard for MRSA tion(s) and the micro-organism in question. Outbreaks outbreaks, SO-ZI/AMR was used [8]. It provides infor- that need to be reported to SO-ZI/AMR are defined as: mation on official outbreaks including the date of the Outbreaks which influence, or have the potential to nega- outbreak and the region and type of health care facility tively influence, access to care, such as in case of (possible) (e.g. hospital or nursing home). The geographical scope closure of a department or part of it, and/or outbreaks of the study was The Netherlands; therefore, we only with continuous transmission despite (infection) control searched using the Dutch language. measures [4]. When reporting outbreaks became part of the profes- sional guidelines, it became essentially mandatory. How- Social media sources and online search behaviour ever, reporting outbreaks is currently only mandatory for To capture social media posts about MRSA (all publicly hospitals, and not for nursing homes or other health shared social media posts by individuals or organiza- care institutions. Once reported, an outbreak is, with a tions), we used Coosto, a social media monitoring tool short delay, immediately visible for all users. The task of [9]. This tool has proven to be a valuable source of social SO-ZI/AMR is not only to collect reports and report media information and is currently in use by the Dutch outbreaks to professionals, but also to monitor the de- government to monitor the quality of healthcare organi- velopment of the outbreak, and, if needed, to support zations [7]. It provides the exact time and, if available, the control efforts. Still, the alert messages from some the location of the message in various social media hospitals seem to come late or not at all, with the risk of sources, including Facebook and Twitter. Presently, its introducing patients infected with an MDRO into a new database includes posts in the Dutch language. healthcare setting without warning and increasing the In addition, Google Trends was used to assess online probability of spreading the outbreak. Therefore, there is search behaviour [10]. This tool provides insight into the a need for additional sources of information about search behaviour based on specific searches performed current and potentially upcoming outbreaks, to increase in Google Search. It provides the relative frequency of outbreak preparedness. searches for different countries and regions. Google Since an increasing number of people use social Trends has been used for early detection of influenza media, such as Facebook and Twitter, to share informa- outbreaks [6]. Although multiple search engines are be- tion and the Internet as source for news, social media ing used in The Netherlands, we limited our searches to posts and online search behaviour (e.g., via search Google Trends since Google covers over 80% of Internet van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 3 of 10 searches, capturing the overall majority of Internet outbreak occurred at least once every 3 months, we used searches [11]. the mean search interest of 3 months in our analyses. We extracted search interest per day, as well as geo- Data extraction graphical information (province) for days with potential We searched Coosto for publicly available social media outbreaks. posts about MRSA with the following search query: (“mrsa” OR “methicillin-resistant Staphylococcus aureus” Statistical analysis OR “methicillin resistant Staphylococcus aureus” OR All statistical analyses were done in SPSS version 22. “meticilline-resistente Staphylococcus aureus” OR “meti- The database with search results from Coosto and Goo- cilline resistente Staphylococcus aureus”). Several prelim- gle Trends was compared to data from the official inary searches revealed that both Dutch and English SO-ZI/AMR database. In case of consecutive days with names had to be included, since English terminology the same potential outbreak, we defined this as a single was sometimes used in Dutch social media posts and the outbreak, both for Coosto and Google Trends. To assess combination maximized the number of hits. Further- the validity of Coosto to detect potential MRSA out- more, the word ‘outbreak’ was not included in the search breaks, we calculated the overall sensitivity, specificity, query, since the preliminary searches showed that positive predictive value (PPV), and negative predictive searching for ‘outbreak’ resulted in a large number of ir- value (NPV) with 95% confidence intervals (CIs). Fur- relevant hits, and that combining ‘outbreak’ with MRSA thermore, we stratified the analyses by type of healthcare via Boolean search (“AND”) negatively affected the sen- institution affected. sitivity of the search. Based on the results of the prelim- For the Google Trends data, a search score was calcu- inary searches, we performed manual inspections of lated for each day: results with 25, 15 and 10 hits per day. In general, the Google Trends Score = (relative search volume - mean lowest number of hits per day would result the highest relative search volume) / standard deviation. number of potential outbreaks. However, from the above with the mean relative search volume and standard de- comparison, we concluded found that 10 posts per day viation based on the preceding 3 months. Using SO-ZI/ was the minimum number of hits, or ‘critical mass’ [7] AMR as the reference standard, we calculated the area to identify a potential MRSA outbreak, Consequently, under the curve (AUC) for 7 days before until 7 days we set (≥10 hits) as criterion for a potential outbreak. after an outbreak. Furthermore, as the optimal cut-off For all posts on a specific day, that met this criterion, we value to detect an outbreak based on online search be- identified whether a potential outbreak was discussed, haviour is unknown, we determined the Google Trends meaning that MRSA was mentioned in relation to a Score that maximized the sum of sensitivity and specifi- Dutch healthcare institution and indicating a present or city. For the present study, we used a cut-off value of potential outbreak. The latter could consist of (but not 2*SD to detect a potential outbreak with Google Trends. limited to) patients, relatives or employees found or sus- Finally, we calculated the Pearson correlation coefficient pected with MRSA, hospital wards closed due to MRSA, any other information about a present or expected Table 1 Characteristics of MRSA outbreaks detected in SO-ZI/ MRSA outbreak shared by the institution, its employees, AMR and Coosto government, or other any other individual (e.g., patients SO-ZI/AMR Coosto Google Trends or relatives). Days meeting all criteria were marked as Number of days with ≥10 posts 297 ‘representing a potential MRSA outbreak’ and all other Excluded 260 days as ‘not representing a potential MRSA outbreak’. Multiple days 22 Dates, number of hits, institution and geographical area and outbreak (YES/NO) were subsequently stored in a New research 33 research database. Unrelated messages 59 Regarding the searches in Google Trends, we used General info concerning MRSA 115 similar terms, but with individual searches for each term. Articles about hygiene and meat 31 Google Trends presents search interest of topics on a Total number of outbreaks 49 37 24 scale from 0 to 100 per day instead of the absolute num- University hospital 8 1 ber of searches, thus every search will have at least one day with the maximum score of 100, even when absolute General hospital 22 17 search numbers are low during a particular time period Nursing home 7 16 (e.g., a time period without any outbreaks). This system Other 3 3 characteristic required a different way of defining days Unknown 9 0 with potential outbreaks. Assuming that a Dutch MRSA van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 4 of 10 Fig. 1 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations to assess the association between the number of posts social media posts, of which 1 referred to an academic related to MRSA detected with Coosto and the Google hospital, 17 to a general hospital, 16 to a nursing home, Trends Score. and 3 to other types of institutions. Google Trends re- sulted in 24 potential outbreaks. Results Figures 1, 2, 3, 4, 5, 6, 7, 8, and 9 show the information During the research period of 15 months (455 days), 49 on MRSA outbreaks originating from the three data outbreaks were reported to SO-ZI/AMR (Table 1). Using sources in each quarter of a year. In only 4 outbreaks Coosto, 37 potential outbreaks were detected based on did all three sources show a (potential) outbreak, with in Fig. 2 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 5 of 10 Fig. 3 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations general the online sources detecting the outbreak 1– by Google Trends. Correlation between the number of 2 days before the official notification date in SO-ZI/ posts related to MRSA detected with Coosto and the AMR. In 48 cases, Coosto and/or Google Trends indi- Google Trends Score was 0.42, p < 0.001). cated a (potential) outbreak without notification in Validity comparisons for Coosto-detected MRSA out- SO-ZI/AMR, whereas in 41 cases, MRSA outbreaks breaks showed an overall sensitivity of 0.20 (95% CI 0.10– were notified to SO-ZI/AMR that went unnoticed by the 0.34) and an overall specificity of 0.96 (95% CI 0.95–0.98), online data sources. In 4 cases, a (potential) outbreak whereas the PPV and NPV were 0.27 (95% CI 0.16–0.42) was detected by both SO-ZI-AMR and Coosto, and not and 0.95 (95% CI 0.92–0.96), respectively (Table 2). After Fig. 4 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 6 of 10 Fig. 5 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations stratification for type of healthcare institution, sensitivity 0.63 (95% CI 0.54–0.73) for MRSA outbreaks in hospi- ranged between 0 for other and unknown locations to tals. With the optimal cut-off for the Google Trends 0.23 (95% CI 0.08–0.45) for general hospitals. Specificity Score, sensitivity was higher for any outbreak compared was ≥0.98 for all types of institutions. with hospital outbreaks only (0.90 vs. 0.43), whereas spe- The validity comparisons for Google Trends to detect cificity was higher for hospital outbreaks (0.28 vs. 0.79). MRSA outbreaks compared with SO-ZI/AMR as the ref- The AUC based on the Google Trends Score 1 day be- erence standard are shown in Table 3. On the exact date fore the official notification was similar to the AUC on the outbreak was notified to SO-ZI/AMR, the AUC was the day of notification. On the other days relative to the 0.59 (95% CI 0.51–0.67) for any MRSA outbreak and notification of the outbreaks, the AUC was decreased. Fig. 6 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/ AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 7 of 10 Fig. 7 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/ AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations Discussion These promising findings suggest that supervisory bod- Principal findings ies such as SO-ZI/AMR may enrich their palette of data In this study, we compared information about potential sources with more dynamic information from social MRSA outbreaks retrieved from social media posts and media and other online sources such as search engine online search behaviour in The Netherlands to the na- data. However, the validity of the online sources Coosto tional notification reference standard. We found that and Google trends needs further investigation. Some simple online (social media) searches do provide add- things need to be discussed. itional information about potential MRSA outbreaks in The sensitivity of the social media monitoring tool The Netherlands compared to the reference standard. Coosto to detect MRSA outbreaks was low and therefore a Fig. 8 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 8 of 10 Fig. 9 MRSA outbreaks in The Netherlands, per quarter of a year. The blue line is the number related to MRSA detected by Coosto, with a red dot indicating days with ≥10 posts related to an outbreak, the red line the Google Trends score. The vertical bars represent outbreaks reported to SO-ZI/AMR: blue for teaching hospitals, black for regular hospitals, green for nursing homes, yellow for other locations, and red for unknown locations substantial number of true outbreaks will be missed when more reluctant to post a message about the outbreak on so- relying on this data source. However, its specificity was cial media than to search for information online. In high, indicating a relatively small number of false positive addition, it is impossible to distinguish between online outbreaks detected by Coosto. Interestingly, the opposite searches for actual outbreaks and searches for random is- was observed for Google Trends, with a higher sensitivity sues related to MRSA using Google Trends. and lower specificity, indicating that Google Trends detects To the best of our knowledge, this was the first study more potential MRSA outbreaks but that many of these using online search engines for social media posts and represent false positive signals. This difference between the internet search behaviour on MRSA outbreak detection. two online data sources may be explained by their nature: A study by Lui et al. used search terms from the social social media posts provide more detailed information on media platform Baidu to identify Noro virus epidemics MRSA outbreaks than online searches, but the patients and [12]. They concluded that several limitations exist to healthcare workers involved in MRSA outbreaks may be using Internet to monitor epidemics but that it still Table 2 Validity comparisons for Coosto-detected MRSA outbreaks with SO-ZI/AMR as the reference standard MRSA outbreaks Number of days Validity TP FP FN TN Se (95% CI) Sp (95% CI) PPV (95% CI) NPV (95% CI) All 10 27 39 722 20 (10–34) 96 (95–98) 27 (16–42) 95 (92–96) Any hospital 5 13 25 755 17 (6–35) 98 (97–99) 28 (13–50) 97 (96–97) University hospital 0 1 8 791 0 100 (99–100) 0 99 (99–99) General hospital 5 12 17 764 23 (8–45) 98 (97–99) 29 (14–52) 98 (97–98) Any hospital and unknown location 6 12 33 747 15 (6–31) 98 (97–99) 33 (17–56) 96 (95–96) University hospital and unknown location 0 1 17 783 0 100 (99–100) 0 98 (98–98) General hospital and unknown location 6 11 25 756 19 (7–37) 99 (97–99) 35 (18–58) 97 (96–97) Nursing home 1 15 6 778 13 (0–53) 98 (97–99) 6 (1–31) 99 (99–99) Nursing home and unknown location 2 14 14 770 13 (2–38) 98 (97–99) 13 (3–37) 98 (98–99) Other location 0 3 3 795 0 100 (99–100) 0 100 (100–100) Other location and unknown location 0 3 12 786 0 100 (99–100) 0 99 (98–99) CI confidence interval, FN false negative, FP false positive, NPV negative predictive value, PPV positive predictive value, Se sensitivity, Sp specificity, TN true negative, TP true positive van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 9 of 10 Table 3 Validity comparisons for Google Trends to detect MRSA outbreaks with SO-ZI/AMR as the reference standard. Day 0 indicates the day the outbreak was reported to SO-ZI/AMR Day Any MRSA outbreak MRSA outbreak in hospital a a AUC (95% CI) Optimal cutoff Se Sp AUC (95% CI) Optimal cutoff Se Sp −7 0.61 (0.54–0.69) −0.2018 75 47 0.59 (0.51–0.67) − 0.6507 100 23 −6 0.51 (0.44–0.59) − 0.1208 63 51 0.51 (0.42–0.59) −0.6965 93 21 −5 0.46 (0.38–0.55) −0.4318 71 33 0.48 (0.39–0.58) −0.8801 97 13 −4 0.50 (0.42–0.58) −0.5524 81 28 0.53 (0.44–0.62) −0.5524 87 28 −3 0.46 (0.38–0.54) −0.5377 81 29 0.48 (0.39–0.58) −0.5377 87 29 −2 0.52 (0.44–0.61) 0.1445 42 65 0.47 (0.37–0.57) −0.6828 83 21 −1 0.59 (0.50–0.67) 0.0425 60 61 0.59 (0.47–0.71) 0.0672 67 62 0 0.59 (0.51–0.67) −0.5524 90 28 0.63 (0.54–0.73) 0.3316 43 79 + 1 0.54 (0.46–0.63) −0.1769 63 48 0.49 (0.39–0.59) −1.1493 100 7 + 2 0.49 (0.41–0.57) − 1.1833 100 6 0.46 (0.36–0.55) −0.9484 97 11 +3 0.50 (0.43–0.58) −0.5149 79 29 0.56 (0.46–0.66) −0.1118 63 51 + 4 0.49 (0.41–0.57) −0.6828 90 21 0.49 (0.40–0.59) −0.6828 93 21 + 5 0.44 (0.37–0.51) 0.1674 15 65 0.39 (0.31–0.47) −1.3370 100 4 + 6 0.49 (0.42–0.57) − 1.0128 98 9 0.47 (0.38–0.57) −1.0128 97 9 + 7 0.53 (0.45–0.60) −0.4993 85 30 0.50 (0.40–0.61) − 0.5745 83 27 AUC area under the curve, CI confidence interval, Se sensitivity, Sp specificity Maximizes sensitivity+specificity. Value represents k might have value as additional tool, particularly when outbreaks to specific organizations, since it does not other monitoring systems are lacking. Also, the importance provide specific names or locations. A refinement of of social media as an early warning system in addition to search mechanism of the freely available default Google traditional slow reporting mechanisms has been empha- Trends software might allow an increase of its sensitivity sized [13]. In general, this might be the case for all out- and specificity. breaks as notification is only done after firm confirmation The extent of information patients and caregivers get of the outbreak. Consequently, the notification date in when confronted with an outbreak may influence their SO-ZI/AMR is “delayed” by several days. search behaviour. If the information is complete and of- Additional potential outbreaks were found via social fered right away, as it is customary in many Dutch hospi- media, which were not in the SO-ZI/AMR database. Most tals, it may become more difficult to detect the outbreak of these outbreaks occurred in nursing homes for which no- using social media and search engine behaviour. tification is not mandatory, but on occasion even hospitals were shown as non-compliant in reporting outbreaks. The Conclusions fact that more dynamic data sources could have value com- Despite several limitations including limited validity, pared to traditional slow reporting mechanisms has also using information from social media and online search been recognized in public health, where social media are behaviour results to detect MRSA outbreaks could be an used as an early warning system for disease outbreaks [14]. additional source of information for supervising bodies, particularly when combined in an automated system Strengths and limitations with real-time updates. The main strength of this study is the use of dynamic Funding online content from social media and search engine be- This research was supported by the EPI-Net COMBACTE-MAGNET project. We haviour in combination with an official reference thank the Innovative Medicines Initiative Joint Undertaking for supporting (SO-ZI/AMR). Using predefined selection criteria, this the EPI-Net COMBACTE-MAGNET project (grant agreement number 115737), resources of which include financial contribution from the European Union allowed us to efficiently study the value of social media Seventh Framework Programme (FP7/2007–2013) and European Federation and online search behaviour via both Coosto and Google of Pharmaceutical Industries and Associations companies in-kind contribution. Trends. Coosto on occasion is limited by the fact that it The funder had no role in the study design, data collection, analysis, interpretation of data or writing the manuscript. may not always be possible to determine whether a po- tential outbreak is actually a true outbreak. Google Availability of data and materials Trends has even more difficulty in this determination, The dataset used during the current study are available from the corresponding for using this data source, it is hard to link potential author on reasonable request. van de Belt et al. Antimicrobial Resistance and Infection Control (2018) 7:69 Page 10 of 10 Authors’ contributions 12. Liu K, Huang S, Miao ZP, Chen B, Jiang T, Cai G, Jiang Z, Chen Y, Wang Z, TvdB, MvG and AV designed the study, with input from LE and JL and RS. Gu H, Chai C. Jiang J identifying potential norovirus epidemics in China via TvdB, PvS, KS and MvG collected and analyzed data. TvdB produced a draft internet surveillance. J Med Internet Res. 2017;19(8):e282. of the manuscript, and LE, JL, RS, JRB, ET, KS, MvG and AV reviewed it at 13. Denecke K, Dolog P, Smrz P. Making use of social media data in public various stages to its final version. All authors read and approved the final health. In: Proceedings of the 21st international conference on world wide manuscript. web. Mountain View: ACM. p. 243–6. 14. Denecke K. Health web science, health information science, chapter 11, social media for health monitoring p 101–108. Switzerland: Springer International Ethics approval and consent to participate Publishing; 2015. https://doi.org/10.1007/978-3-319-20582-3_11. Since the anonymous data used in this study were derived from the public social media domain without patient involvement, no medical ethical review was needed in the Netherlands. SO-ZI/AMR allowed us to use anonymous (not linked to specific hospital organizations) information from their database. Competing interests AV is Editor-in-Chief of ARIC. All other authors declare that they have no competing interests. Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Author details Radboud REshape Innovation Center, Radboudumc University Medical Center, Geert Grooteplein Zuid 10, 6525, GA, Nijmegen, Netherlands. 2 3 VetEffecT, Bilthoven, The Netherlands. Unidad Clínica de Enfermedades Infecciosas y Microbiología Instituto de Biomedicina de Sevilla (IBiS) /Hospital Universitario Virgen Macarena / CSIC / Departamento de Medicina, Universidad de Sevilla, Sevilla, Spain. Division of Infectious Diseases, Tübingen University Hospital, DZIF Center, Tübingen, Germany. Infectious Diseases, University of Verona, Verona, Italy. Department of Medical Microbiology, Radboudumc, Nijmegen, The Netherlands. Department for Health Evidence, Radboud Institute for Health Sciences, Radboudumc, Nijmegen, The Netherlands. Department of Clinical Microbiology and Infectious Diseases, Canisius-Wilhelmina Hospital, Nijmegen, The Netherlands. Received: 15 January 2018 Accepted: 15 May 2018 References 1. WIP-guideline MRSA. https://www.rivm.nl/dsresource?objectid=3f054354-ff4a- 43ef-91f9-7c6f0417be95.Accessed 30 Aug 2017. 2. WIP-guideline BRMO (Bijzonder Resistente Micro-Organismen). https://www. rivm.nl/dsresource?objectid=b6b99580-44e2-4b9c-8183-52871e61764f&type= org&disposition=inline. Accessed 30 Aug 2017. 3. Van der Bij AK, van Dijk K, Muilwijk J, Thijsen SFT, Notermans DW, de Greeff S. Van de Sande-Bruinsma N. Clinical breakpoint changes and their impact on surveillance of antimicrobial resistance in Escherichia coli causing bacteraemia. Clin Microbiol Infect. 2012;18:E466–74. 4. RNVWS.nl Criteria uitbraak. https://www.nvmm.nl/zorg/uitbraakmeldingen/. Accessed 30 Aug 2017. 5. Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L. Detecting influenza epidemics using search engine query data. Nature. 2009;457:1012–4. 6. Carneiro HA, Mylonakis E. Google trends: a web-based tool for real-time surveillance of disease outbreaks. Clin Infect Dis. 2009;49:1557–64. 7. van de Belt TH, Engelen LJ, Verhoef LM, van der Weide MJ, Schoonhoven L, Kool RB. Using patient experiences on Dutch social media to supervise health care services: exploratory study. J Med Internet Res. 2015;17(1):e7. 8. van der Bij AK, Kardamanidis K, Frakking FN, Bonten MJ. Signaleringsoverleg Ziekenhuisinfecties en Antimicrobiële Resistentie. Nosocomial outbreaks and resistant microorganisms. Ned Tijdschr Geneeskd. 2015;159:A8585. 9. Coosto, Social Media Monitoring, Webcare & Publishing. Coosto B.V. Eindhoven, The Netherlands. Version 2017 Coosto http://www.coosto.nl. Accessed 30 Aug 2017. 10. Google Trends. Google. http://google.com/trends. Accessed 30 Aug 2017. 11. Nationale Search Engine Monitor (2014) (iProspect en RMI) https://www. iprospect.com/nl/nl/onze-blog/nationale-search-engine-monitor-2014/. Accessed 30 Aug 2017.

Journal

Antimicrobial Resistance & Infection ControlSpringer Journals

Published: May 30, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off