Access the full text.
Sign up today, get DeepDyve free for 14 days.
Loading next page...
/lp/ou_press/implementation-evaluation-and-recommendations-for-extension-of-ahrq-HPSaR090Yu
- Copyright
- © The Author(s) 2018. Published by Oxford University Press on behalf of the American Medical Informatics Association.
- eISSN
- 2574-2531
- DOI
- 10.1093/jamiaopen/ooy004
- Publisher site
- See Article on Publisher Site
Abstract
Downloaded from https://academic.oup.com/jamiaopen/article-abstract/1/1/20/4959057 by Ed 'DeepDyve' Gillespie user on 07 November 2018 JAMIA Open, 1(1), 2018, 20–25 doi: 10.1093/jamiaopen/ooy004 Advance Access Publication Date: 3 April 2018 Brief Communication Brief Communication Implementation, evaluation, and recommendations for extension of AHRQ Common Formats to capture patient- and carepartner-generated safety data 1,2 3 3,4 Sarah Collins, RN, PhD, Brittany Couture, Patricia Dykes, RN, PhD, Jeffrey 3,4 3 5 Schnipper, Maureen Fagan, DNP, MHA, James Benneyan, PhD, Aziz Sheikh, MD, 6 3,4,7 3,4 MSc, David W. Bates, MD, MSc, and Margarita Sordo, MSc, PhD 1 2 Columbia University, Department of Biomedical Informatics, New York, NY, Columbia University, School of Nursing, New York, 3 4 NY, Brigham and Women’s Hospital, Center for Patient Safety, Research and Practice, Boston, MA, Harvard Medical School, 5 6 Boston, MA, Northeastern University, HealthCare Systems Engineering Institute, Boston, MA, Usher Institute of Population Health Sciences and Informatics, The University of Edinburgh, Scotland, UK and Harvard School of Public Health, Boston, MA Corresponding Author: Sarah Collins, RN, PhD, Assistant Professor of Biomedical Informatics and Nursing, Columbia University Medical Center, 622 West 168th St, PH 20th Floor, New York, New York 10032, USA (sac2125@cumc.columbia.edu) Received 11 January 2018; Revised 16 February 2018; Accepted 4 March 2018 ABSTRACT Objectives: The Common Formats, published by the Agency for Healthcare Research and Quality, represent a standard for safety event reporting used by Patient Safety Organizations (PSOs). We evaluated its ability to cap- ture patient-reported safety events. Materials and methods: We formally evaluated gaps between the Common Formats and a safety concern reporting system for use by patients and their carepartners (ie friends/families) at Brigham and Women’s Hospital. Results: Overall, we found large gaps between Common Formats (versions 1.2, 2.0) and our patient/carepartner reporting system, with only 22–30% of the data elements matching. Discussion: We recommend extensions to the Common Formats, including concepts that capture greater detail about the submitter and safety categories relevant to unsafe conditions and near misses that patients and care- partners routinely observe. Conclusion: Extensions to the Common Formats could enable more complete safety data sets and greater un- derstanding of safety from key stakeholder perspectives, especially patients, and carepartners. Key words: patient safety, safety reporting, patient-generated health data, standards data source. Patient-generated health data that captures INTRODUCTION patient-reported safety concerns could be useful to increase the Safety reporting systems are essential for understanding patient completeness of patient safety event data by including the patient safety issues and supporting a “Just Culture” of learning from ad- and carepartner (eg family members or friends involved in a patient’s 1–3 verse events for continuous improvement. Most safety data used care) perspective of what comprises an unsafe condition, near miss or for safety and quality improvements within health organizations incident, and how frequently those events are experienced. and across Patient Safety Organizations (PSOs) are clinician- Capturing safety concerns from patients in real-time is still in its reported, and underestimate true harm rates. Patient-generated infancy, with only a few known applications that have been devel- 7,8 health data (PGHD) increasingly are recognized as an important oped and piloted. Standard formats are necessary to leverage V The Author(s) 2018. Published by Oxford University Press on behalf of the American Medical Informatics Association. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unre- stricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. 20 Downloaded from https://academic.oup.com/jamiaopen/article-abstract/1/1/20/4959057 by Ed 'DeepDyve' Gillespie user on 07 November 2018 JAMIA Open, 2018, Vol. 1, No. 1 21 information and knowledge from safety event data pooled across clinical patient safety experts for mapping and interpretation of pa- individuals, settings, and health organizations. Capturing safety tient/carepartners reported safety data and concepts (authors: S.C., event data from patient/carepartner perspectives in standard formats J.S., and P.D.); and (3) iterative confirmation of mapping with team will allow cross-referencing with clinician-reported data to compare member experienced in information modeling for validation of se- perspectives, reporting rates, and track individual events or types of mantic consistency and modeling approach (authors: M.S., S.C.). In events. Such cross-referencing activities will be much easier if the 2017, AHRQ updated the version 1.2 of the Common Formats data elements used for reporting by different roles (eg patients, care- (v1.2) by releasing version 2.0 (v2.0), based on feedback from the partners, clinicians) map to each other. PSO community, which specified a smaller core data set that will be The Common Formats published by the Agency for Healthcare used for national aggregation and analysis. Analyzes were com- Research and Quality (AHRQ) represent a major step forward and pleted for both v1.2 and the v2.0. are the standard used and endorsed by PSOs to enable sharing and During the mapping and analysis steps described above, we learning from de-identified data sets of safety events that occur in attempted to model MySafeCare data elements as comprehensively the clinical setting. The Common Formats standard primarily tar- as possible without risking the integrity of the data, as judged by an gets clinician-reported safety events and captures 3 types of safety information modeling expert (M.S.). We identified modifications to events: unsafe conditions, near misses, and incidents. The Common MySafeCare when it would improve data capture without antici- Formats defines an “incident” as a safety event that reached the pa- pated negative impacts on patient/carepartner usability based on our tient, “a near miss” as a safety event that did not reach the patient, user-centered design findings. To quantify the gap between MySafe- and an “unsafe condition” as a circumstance that increases the prob- Care and the Common Formats, we used codes consistent with Fifth ability of a safety event. While the Common Formats standard does Message Understanding Conference (MUC-5) Evaluation Metrics to not exclude patient-reported safety data from being modeled to ad- categorize each MySafeCare data element as match, partial match, here to its format, we believe significant gaps serve as barriers to us- conflicting, extra, or missing from the Common Formats. Finally, ing this standard to adequately capture safety events reported by we implemented a Clinical Document Architecture (CDA) XML patients and their carepartners. and schema using the Common Formats generic and medication Problems with the validity and reliability of clinician-reported modules from quality reporting CDA release 2.0 using sample safety data are well-documented; even still, clinicians’ professional MySafeCare data and provide screen shots. training likely provides a baseline knowledge and literacy of patient safety concepts that should inform some level of accuracy and reli- RESULTS ability in safety reporting. Patients and their carepartners, as a group, do not receive comparable professional training. Therefore, The revised v2.0 of the Common Formats resulted in a decrease in content and functional specifications for a patient/carepartner the number of data elements fields within MySafeCare that could be reporting system will vary from content and functional specifica- mapped using the standard from 30% in v1.2 to 22% in v2.0 (see tions for clinician reporting systems. These variations require evalu- Figure 1). Partial matches also decreased from 11% to 8% with the ation of current standards capability to adequately capture patient/ release of v2.0. The number of MySafeCare fields that could not be carepartner-reported safety events to enable aggregation with safety mapped to the Common Formats (ie missing) increased from 58% data from other sources (eg clinicians). in v1.2 to 69% in v2.0. We used the Common Formats to model the data captured in an Data elements to capture “submitter and submission existing electronic patient/carepartner safety reporting system used information” dropped from 5 of 9 matching or partial match in 10,11 at Brigham and Women’s Hospital called MySafeCare. MySafe- Common Formats v1.2 to only 1 data element matching in v2.0. Care was developed based on other safety reporting literature and Matches for v2.0 and v1.2 to MySafeCare remained the same for 3,12,13 systems and used an iterative user-centered design process. Concern Categories, but level of match for subcategories and demo- MySafeCare includes features such as patient/carepartners’ ability to graphics decreased. For example, we could no longer capture that submit anonymously and provides 9 safety concerns categories for data were submitted anonymously, the submitters relationship to user selection (see Table 1). To promote use of MySafeCare in the the patient (eg is the patient, or is a family member), and some an- hospital, in-person engagement rounds are conducted with patient/ swer choices such as “other” gender (ie not male, female, or un- carepartners’ explaining “we would like to hear from you through known). MySafeCare to improve reporting of concerning or worrisome We identified the need to disambiguate 2 MySafeCare data ele- events.” ments, an incident and a near miss, in which “wrong medication” We evaluated and quantified the gaps between Common For- and “wrong dose” were combined in the answer choices (see mats and MySafeCare, and drawing on this analysis we sought to Table 1). The CDA schema can be seen in Figures 2 and 3. make recommendations for extensions to the standard to capture patient-/carepartner-reported safety data. This work is part of a large study investigating patient/carepartner safety reporting which DISCUSSION has been approved by our organization’s Institutional Review Board. Release of v2.0 of AHRQ Common Formats decreased the total number of data elements in the Common Formats significantly, and our analyzes indicated that this resulted in decreased capability to map MySafeCare data elements. Overall, we identified a METHODS “narrowing” of concepts captured from v1.2 to v2.0 that further re- The study team utilized 3 steps to validate mapping of concepts stricted the reporting options that were available in v1.2 for non- from MySafeCare into AHRQ Common Formats: (1) individual clinical people (ie patient/carepartner) to report a safety issue. While mapping of concepts (author: S.C.); (2) group consensus with v1.2 could not map all MySafeCare concepts, it included more Downloaded from https://academic.oup.com/jamiaopen/article-abstract/1/1/20/4959057 by Ed 'DeepDyve' Gillespie user on 07 November 2018 22 JAMIA Open, 2018, Vol. 1, No. 1 Table 1. MySafeCare data elements with match, partial match, or Table 1. continued missing in AHRQ Common Formats v1.2 and 2.0 MySafeCare data elements AHRQ Common Formats MySafeCare data elements AHRQ Common Formats V1.2 V2.0 V1.2 V2.0 My medical device seems excessive Missing Missing My room is not clean Missing Missing Metadata I/my carepartners have other Missing Missing Organization OID Match Match medical device concerns PSO ID Match Match I/my carepartners do not feel Missing Missing Subject ID Match Match respected Submission date/time Match Match My and/or my carepartners needs Missing Missing Clinical unit Match Match are being ignored Submitter and submission information I/my carepartners am/are Missing Missing Would you like to tell us your Match Missing concerned about the name and room number or stay communication between my anonymous? care team about my plan of care What is your name? Match Missing I/my carepartners am/are Missing Missing What is your room number? Missing Missing concerned about how my care What is your relationship with the Match Missing team communicates with me patient? about my plan of care Is your family/friend engaged in Missing Missing I/my carepartners have other Missing Missing your care? communication concerns On a scale of 1–3, with 3 being the Missing Missing A member of my care team did not Missing Missing most worried, how worried are wash his/her hands you about the unexpected or A member of my care team did not Missing Missing concerning event(s) that you wear gloves experienced? A member of my care team did not Missing Missing When did this unexpected or Match Match follow the precautions on the concerning event occur? door Did you share your concern with Partial Missing I/my carepartners have other Missing Missing your care team? infection concerns Do you plan to share your Missing Missing My and/or my carepartners Missing Missing concern? privacy is/are being ignored Concern categories I/my carepartners have other Missing Missing My plan Partial Partial privacy concerns My communication Partial Partial I/my carepartners feel that my care Missing Missing My privacy Partial Partial team is not managing my pain My pain Partial Partial to my expectations My waiting time Partial Partial My pain is well controlled but I/ Missing Missing My medication Match Match my carepartners am/are My room Match Match concerned about the medication My hygiene Match Match Nobody has asked me or my Missing Missing Other Match Match carepartners if I am in pain Concern subcategories I/my carepartners have other pain Missing Missing I/my carepartners do not know my Missing Missing management concerns plan of care I am waiting too long for help Missing Missing I/my carepartners feel like my care Missing Missing going to the bathroom team is not following my plan I am waiting too long for help Missing Missing of care turning and moving in bed I/my carepartners feel there is a Missing Missing I am waiting too long for my Missing Missing problem with my plan of care procedure I/my carepartners have other Missing Missing I am waiting too long to be Missing Missing treatment concerns transferred I was given the wrong medication Partial Missing I am waiting too long to be Missing Missing or dose discharged I was almost given the wrong Partial Missing I/my carepartners have other Missing Missing medication or dose waiting time concerns I was not given my medication Match Match Narrative content on-time Please describe the event in your Match Match I missed a medication Match Missing own words: I/my carepartners have other Match Missing Follow-up documented by Match Missing medication concerns clinician My medical device is not working Missing Missing Optional background information My medical device will not stop Missing Missing What is your age? Match Match beeping (continued) (continued) Downloaded from https://academic.oup.com/jamiaopen/article-abstract/1/1/20/4959057 by Ed 'DeepDyve' Gillespie user on 07 November 2018 JAMIA Open, 2018, Vol. 1, No. 1 23 Table 1. continued MySafeCare data elements AHRQ Common Formats V1.2 V2.0 What is your gender? Match Partial If other, please specify: Missing Missing Was the admission to the hospital Missing Missing planned or urgent/emergent? What is your race? (please choose Match Match one or more) If other, please specify: Missing Match Are you Hispanic or Latino? Match Match What is your preferred language? Missing Missing If other, please specify: Missing Missing What is your ZIP code? Missing Missing What is the highest level of Missing Missing education you have completed? How often do you need to have Missing Missing someone help you when you read instructions, pamphlets, or other written material from Figure 2. General view of the medication Clinical Document Architecture your doctor or pharmacy? showing header and 2 sections: generic and medication. Count of extra fields from AHRQ 189 80 Common Formats not present in MySafeCare For example, clinician-reported incident of a late (or early) medica- tion is likely only submitted when it is clinically significant and po- Recommended revisions to MySafeCare to align with Common Formats. tentially harmful (eg antibiotic prophylaxis for prevention of surgical site infection). A patient-reported concern that a medication was late may not consider that the administration time was still clin- ically appropriate (eg daily aspirin received an hour late). Impor- tantly, we still want to capture all patient/carepartners’ concerns, but need to ensure our reporting standards allow for both differenti- ating and pooling these data appropriately. AHRQ Common Formats can have one or more event categories in a report (fall event and medicine event), but there are constraints associated with each report type (near miss, incident, or unsafe con- dition). For example, a fall event should not be included in a near miss or unsafe condition report, whereas a medication event may be included in an incident, near miss or unsafe condition report. Addi- tionally, MySafeCare captured the “date/time” of the submission (which can be captured using the CDA in the generic module) but Figure 1. Agency for Healthcare Research and Quality Common Formats v1.2 MySafeCare also captures relative dates by asking if the event oc- and 2.0 Mapping to MySafeCare Patient and Care Partner Reporting System. curred today, yesterday, or more than 2 days ago. The phrasing of these questions was carefully considered and tested for usability and options than v2.0. Overall, only 22% of concepts could be mapped comprehension by hospitalized patients. However, the CDA cap- with the latest version, suggesting that the standard needs to be tured more “punctual” events, rather than “events in an interval” as extended. MySafeCare did, since it covered relative dates within a full hospital The Common Formats include a generic module and an event stay. Therefore, we chose to use one report per event, given report specific module, with the generic section required for all types of type constraints and that all events are unlikely to happen on the reports. We found that all concepts in our patient/carepartner safety same date. This approach provided more specific capture of meta- reporting application that could be modeled using the Common For- data per safety event, but required extra processing. mats were part of the generic module, except medication concerns. Using a standard that integrates the 2 types of data (clinician- We could model MySafeCare medication concerns using the Com- reported and patient/carepartner-reported) and facilitates interpreta- mon Formats medication module and note that MySafeCare could tions based on data provenance could be beneficial for improved be extended to capture more detailed medication information, such measuring, monitoring, and quality improvement initiatives and as the medication name. MySafeCare was developed for ease of use might lead to more robust signal detection of emerging patterns of by patients/carepartners and to identify types of safety concerns, safety issues. For example, the ability to detect if clinicians and which is why this level of medication detail was not specified, but patients/carepartners identify issues that apply to unique domains may be desired as we seek to align data sources and increase utiliza- could highlight major gaps in current safety initiatives that are only tion of PGHD. However, we note that interpretation of some con- based on clinician-reported data. Alternatively, the ability to identify cerns, particularly medication concerns, submitted by patients/ if clinicians and patients/carepartners identify different issues within carepartners may not be equivalent to a clinicians’ incident report. the same domain could be particularly useful to better understand Downloaded from https://academic.oup.com/jamiaopen/article-abstract/1/1/20/4959057 by Ed 'DeepDyve' Gillespie user on 07 November 2018 24 JAMIA Open, 2018, Vol. 1, No. 1 Limitations This evaluation and modeling is based on the MySafeCare system, developed using user-centered design to capture patient/carepartner perspectives of safety events. Other patient safety reporting systems should be evaluated for further recommendations to extend Com- mon Formats. CONCLUSION Providing patients/carepartners with an easy, electronic mechanism to directly share and record safety concerns and events with clini- cians and hospital administrators in real-time is necessary, but not sufficient, to advance our understanding of safety threats. These data must be captured in a standard format to enable identification of system weaknesses, continuous learning, and evaluation of inter- ventions at scale. Most standards need iterative refinement, and this relatively new standard is no exception. We recommended 4 exten- sions to AHRQ Common Formats to enable PSOs to capture safety events reported from the patient/carepartner perspective to increase completeness and understanding of safety from multiple stakeholder perspectives. Future work should continue to validate relevant data to capture from patients/carepartners to inform patient safety improvements and enable pooling and sharing of safety data sets for continuous learning. FUNDING Funding provided by the Agency for Healthcare Research and Quality (AHRQ) 1P30HS0235335 Making Acute Care More Patient Centered. The Figure 3. Generic section expanded of Clinical Document Architecture content is solely the responsibility of the authors and does not necessarily rep- schema. resent the official views of the Agency for Healthcare Research and Quality. complex domains, such as discharge planning and communication Conflict of interest statement. None declared. that likely require multi-faceted interventions for successful and im- pactful change. CONTRIBUTORS Based on our analyzes, we propose 4 broad recommendations for extension to AHRQ Common Formats Standard with the intent All authors have contributed sufficiently and meaningfully to the to capture more comprehensive safety data inclusive of patient/care- conception, design, and conduct of the study; data acquisition, anal- partners’ perspectives (discussed below). These recommendations ysis, and interpretation; and/or drafting, editing, and revising the should serve as guidance for initial revisions to the Common For- manuscript. mats and should be refined through continued validation with other patient and carepartner safety event reporting systems. ACKNOWLEDGEMENTS We thank the patients and families who informed the content and functional- ity of MySafeCare. Recommendations for extension to AHRQ common REFERENCES Formats standard 1. Extend model to capture whether data reflects patient, carepart- 1. Leape L. Why should we report adverse incidents? J Eval Clin Pract 1999; ner, or clinician reporting to allow for differentiating and pool- 5 (1): 1–4. 2. Wachter RM, Pronovost PJ. Balancing “No Blame” with accountability in ing data appropriately. patient safety. N Engl J Med [Internet] 2009; 36114 (361): 1401–6. 2. Capture more complete information about the submission than 3. Schnall R, Bakken S. Reporting of hazards and near misses in the ambula- is captured in current versions of the standard, including if data tory care setting. J Nurs Care Qual [Internet] 2011; 26 (4): 1. was reported anonymously. 4. Weingart SN, Pagovich O, Sands DZ, et al. What can hospitalized patients 3. Revise value sets to include additional relevant values when an- tell us about adverse events? Learning from patient-reported incidents. swered from patient or carepartner perspective, such as “other” J Gen Intern Med 2005; 20 (9): 830–6. gender. 5. ONC. Patient-Generated Health Data j Policy Researchers & Implement- 4. Extend categories that capture observations from patients and ers [Internet]. HealthIT.gov. 2017. https://www.healthit.gov/policy- carepartners related to safety, particularly categories for events researchers-implementers/patient-generated-health-data. Accessed Octo- types of unsafe conditions and near misses. ber 7, 2017. Downloaded from https://academic.oup.com/jamiaopen/article-abstract/1/1/20/4959057 by Ed 'DeepDyve' Gillespie user on 07 November 2018 JAMIA Open, 2018, Vol. 1, No. 1 25 6. Couture B, Fagan M, Gershanik E, Benneyan J, Bates DW, Collins SA. To- In: AMIA Annual Symposium Proceedings [Internet]. 2015, p. 1. https:// wards analytics of the patient and family perspective: a case study and rec- knowledge.amia.org/59310-amia-1.2741865/t005-1.2744350/f005-1. ommendations for data capture of safety and quality concerns. In: AMIA 2744351/2247185-1.2745295/2247185-1.2745296? timeStamp¼1457 Annual Symposium Proceedings; 2017; Washington, D.C. 468408316. Accessed March 8, 2016. 7. Lawton R, O’Hara JK, Sheard L, et al. Can patient involvement improve 11. Collins S, Couture B, Lilley L, Smith A, Chang F, Gershanik E. Increased patient safety? A cluster randomised control trial of the Patient Reporting understanding of safety concerns experienced by hospitalized patients and and Action for a Safe Environment (PRASE) intervention. BMJ Qual Saf their families through real-time reporting in MySafeCare App. In: AMIA 2017; 26 (8): 622–31. Annual Symposium Proceedings; Chicago, IL. 8. Schneider EC, Ridgely MS, Quigley DD, et al. Developing and Testing the 12. Weissman JS, Schneider EC, Weingart SN, et al. Comparing patient- Healthcare Safety Hotline: A Prototyope Consumer Reporting System for reported hospital adverse events with medical record review: do patients Patient Safety Events. Final Report (Prepared by RAND Corporation un- know something that hospitals do not? Ann Intern Med 2008; 149 (2): der contract HHSA290201000017I) [Internet]. Rockville, MD; 2016 100–8. https://www.ahrq.gov/sites/default/files/publications/files/healthcare- 13. Currie LM, Desjardins KS, Levine ES, et al. Web-based hazard and near- safety-hotline_0.pdf. Accessed November 13, 2017. miss reporting as part of a patient safety curriculum. J Nurs Educ 2009; 9. AHRQ. AHRQ Common Formats - Hospital Version 1.2 [Internet]. 2013. 48 (12): 669–77. https://www.psoppc.org/psoppc_web/publicpages/supportingDocsV1.2 14. Chinchor N, Sundheim B. MUC-5 evalution metrics. In: Fifth Message 10. Couture B, Cleveland J, Ergia A, et al. User-Centered Design of the Clini- Understanding Conference (MUC-5); 1992. p. 69–78. cal Dashboard for the MySafeCare Patient Safety Reporting System.
Journal
JAMIA Open – Oxford University Press
Published: Jul 1, 2018