Implementation of a patient-facing genomic test report in the electronic health record using a web-application interface

Implementation of a patient-facing genomic test report in the electronic health record using a... Background: Genomic medicine is emerging into clinical care. Communication of genetic laboratory results to patients and providers is hampered by the complex technical nature of the laboratory reports. This can lead to confusion and misinterpretation of the results resulting in inappropriate care. Patients usually do not receive a copy of the report leading to further opportunities for miscommunication. To address these problems, interpretive reports were created using input from the intended end users, patients and providers. This paper describes the technical development and deployment of the first patient-facing genomic test report (PGR) within an electronic health record (EHR) ecosystem using a locally developed standards-based web-application interface. Methods: A patient-facing genomic test report with a companion provider report was configured for implementation within the EHR using a locally developed software platform, COMPASS™.COMPASS™ is designed to manage secure data exchange, as well as patient and provider access to patient reported data capture and clinical display tools. COMPASS™ is built using a Software as a Service (SaaS) approach which exposes an API that apps can interact with. Results: An authoring tool was developed that allowed creation of patient-specific PGRs and the accompanying provider reports. These were converted to a format that allowed them to be presented in the patient portal and EHR respectively using the existing COMPASS™ interface thus allowing patients, caregivers and providers access to individual reports designed for the intended end user. Conclusions: The PGR as developed was shown to enhance patient and provider communication around genomic results. It is built on current standards but is designed to support integration with other tools and be compatible with emerging opportunities such as SMART on FHIR. This approach could be used to support genomic return of results as the tool is scalable and generalizable. Keywords: Genomics, Medical informatics applications, Electronic health records, Patient-centered care, Patient access to records, Communication Background challenge for patients and their providers is having ready Precision Medicine with an emphasis on the incorporation access to the information that is necessary for appropri- of genomic information into the clinic has been identified ate management and coordination of care. When faced as a priority for research in the United States [1]. with a patient with a genetic condition they have not Genetic disorders, while individually rare, are collect- previously encountered, knowledge and resources are ively common. It is estimated that there are over 6800 extremely limited. This often puts patients and families rare and ultra-rare disorders, many of which are genetic in the position of attempting to become the ‘expert’ in affecting approximately 30 million Americans [2]. The the specific disease via an unguided internet search process which can lead to uncomfortable and sometimes adversarial encounters with the medical system. * Correspondence: mswilliams1@geisinger.edu One potential solution to lower the barriers experi- Genomic Medicine Institute, Geisinger, 100 North Academy Avenue, enced by providers caring for patients with genetic Danville, PA, USA Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 2 of 9 conditions is using fully functional electronic health rec- recently concluded and demonstrates qualitative im- ord systems (EHR) [3–5]. The capabilities provided by provement in communication and satisfaction. such EHRs, particularly through knowledge management The purpose of this paper is to present the technical systems and clinical decision support systems, have been aspects of development and deployment of the first demonstrated to significantly improve process outcomes, patient-facing genomic test report (PGR) within an EHR although the evidence of impact on health outcomes is ecosystem using a locally developed web-application less robust [6]. interface. While these techniques have mostly faced toward pro- viders, similar approaches could be used to provide in- Methods formation directly to patients and families. A recent A PGR was developed using a mixed-methods formative study conducted at 3 healthcare organizations, including approach which has been previously reported [11]. The Geisinger, studied the impact of opening patient access PGR includes content that was considered important by to their provider notes through a secure patient portal patients and their families. The content is vetted for (OpenNotes) [7]. Using a quasi-experimental trial de- readability through a review process that includes con- sign, several significant differences were noted when tent experts, non-genetic providers and patients and is comparing the pre- and post-intervention surveys in- formally tested to meet accepted literacy standards. The cluding patients reporting that OpenNotes, “…helped PGR was developed using paper mock-ups. Once the them feel more in control of their care” and “…increased final format was chosen, this was converted to an elec- medication adherence:”. A recent article noted that tronic format that could be displayed through an inter- patient-facing applications could enable meaningful use face such as a patient portal, but could also be printed objectives promoted by the Office of the National Co- to provide a hard copy. A similar process was used to ordinator of Health Information Technology (ONCHIT) create a provider version of the PGR that was designed [8]. This approach is also endorsed by the recent report to complement the PGR [12]. These two reports were from the National Academy of Medicine (formerly the then submitted to the COMPASS™ development team Institute of Medicine), Improving Diagnosis in Health for implementation. Care [9] which highlights the importance of patient COMPASS™ is a software platform developed at Gei- empowerment. singer intended to work as an add-on to EHRs. Its main One area that has received less emphasis is the role of purpose is to manage secure data exchange, as well as the laboratory report in communicating information to patient and provider access to patient reported data cap- patients. Haga et al. [10] propose four revisions to ture and clinical display tools utilized within Geisinger. current genomic test reports that could improve accessi- It is intended to improve patient engagement in their bility for patients. These include, “…1) inclusion of an care and enhance patient-provider communication. interpretive summary section, 2) a summary letter to ac- COMPASS™ is built using the ASP.Net framework company the lab report, 3) development of a patient user where the presentation layer (User Interface) imple- guide to be provided with the report, and 4) a com- ments web development languages and libraries includ- pletely revised patient-friendly report.” In genomic medi- ing HTML, CSS, JavaScript, JQuery, AngularJS, and cine, direct-to-consumer companies have emerged that Angular2. The back-end (Code Behind, Logic, Data In- provide results of tests to consumers without the inter- teractions) is coded using C# and utilizes SQL Data mediary of a health care professional. While many of the Adapters, SOAP Web Services, WebAPIs, and Teradata results are non-health related (e.g. ancestry) some com- Data Adapters to facilitate data transfers. The COM- panies are returning predictive information that could PASS™ data model utilizes MS SQL Server but it is not have an impact on an individual’s health. The positive limited to only utilizing SQL Server for interacting with and negative outcomes of providing this information dir- data as it can consume data from web services or any ectly to consumers have not been rigorously studied. standard database platform through its Software as a Our group has previously reported on the design and Service (SaaS) approach. Through this, users can either testing of linked patient- and provider-facing genomic pass data to the COMPASS™API for use or COMPASS™ test reports [11, 12]. These reports are not meant to re- can access common database platforms and consume/ place the laboratory report, but to provide additional in- display appropriate data. It is designed so that any con- formation to guide interpretation and clarify clinical tent can be coded to be accessible on any other resource. recommendations that result from the testing. As such This allows content in COMPASS™ to be agnostic from they are compatible with the recommendation from its own server/resource perspective. Haga to create a “a patient-friendly report” [10]. A com- COMPASS™ is integrated with Geisinger’s primary parative effectiveness study [13] that compares the use EHR (Epic) on several levels. The EHR interacts with of these reports to standard of care communication has COMPASS™ by launching an external web browser via a Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 3 of 9 specific encrypted URL. The current implementation patient specific reports. Through COMPASS™, the pa- within Epic is a visit navigator link which contains re- tient report template interfaces with various data sources quired encrypted parameters for authentication. COM- in addition to the database storing the information gen- PASS™ is fed by real-time data feeds from the EHR to erated from the generic gene template (Fig. 2), inherit- correctly determine project inclusion/exclusion and dis- ance pattern template, and glossary term template. One play correct and current data to users. Currently, the specific tool, the SimulConsult® Genome-Phenome real time feeds that COMPASS™ utilizes are custom and Analyzer (SGPA) [14] has been used to provide informa- standard data extractions that interface with Epic. COM- tion to the report. A local API is called when a user fin- PASS™ is also integrated with patient portals (currently ishes creating the content for the custom report in the MyGeisinger which is a MyChart implementation) so authoring environment. The author then opens the Epic patients can use the application when off-site from the record for the patient and launches the SGPA from the health system network. As with the EHR, COMPASS™ encounter navigator. In the SGPA, patient information is interacts with the portal by launching an external web combined with the genomic sequence information and a browser via an encrypted URL. gene associated with the clinical features along with a An authoring application was created within COM- variant in that gene are selected by the author. This PASS™ to assist geneticists and genetic counselors with ‘closes the loop’ by associating the gene and variant with building coordinated and complementary patient and the specific patient. The author then clicks the “Send family-facing and provider-facing reports of the patient’s XML Report” button in the SGPA which sends Simul- test results. The authoring application includes areas to Consult® content back to the authoring environment. A create and store generic genetic content for the reports custom parser developed to support the project, in com- that is not readily available from other sources. The bination with the SimulConsult® Web API loads the data authoring environment contains reusable content fields into the “PatientReports” table in the Genomics data- (including inheritance patterns, glossary terms and gene base. The final patient-specific report is available for re- templates that create gene-specific content that can be view and publication at which time it appears in the used for all reports that return results related to the MyGeisinger patient portal, and is accessible to clini- gene) that can be selected to populate patient and pro- cians in the patient’s encounter navigator through a local vider reports. The environment tracks versioning of the Web Apps URL. One specific example of SGPA content information created within the authoring application— that was highly valued by both patients and providers an essential element given the rapidly changing nature was the prognosis table. This prognosis table, which of knowledge in genomics that can have implications for summarizes medical issues associated with a condition liability. The ecosystem of the PGR report is presented the likelihood that the issue will develop and when in in Fig. 1. the life of the patient they are likely to appear, is auto- A patient report template is used by the geneticist/ matically generated by the SGPA [14]. The manual cre- counselor to pull information together to create the ation of such a table would require a tremendous Fig. 1 COMPASS™ Data Flow. A diagram of the data flows for COMPASS™. Note that the EHR data flow is in the process of moving to SMART on FHIR which should increase the generalizability of the tool in different EHR ecosystems Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 4 of 9 Fig. 2 The generic gene template. An excerpt of the MECP2 gene template from the COMPASS™ authoring tool investment of time and expertise. While not required it is available to patients and their authorized representa- for the authoring process, use of the SGPA improves tives (usually family members). Patients and their fam- the workflow and reduces the time needed to author ilies can access the report electronically through a link a report. Upon selection and verification of the con- out to COMPASS™ from the patient’s MyGeisinger por- tent, the report content is published from the author- tal or prior to a visit in the clinic during the nurse room- ing application and made available to patients and ing process in Epic (Fig. 3). The PGR is also designed so their families and providers electronically in that a hard copy can be printed—a requirement that was COMPASS™. requested by participants in the formative study. The re- port is written and laid out in a manner based on input Results from the interviews and focus groups that reflects the After the PGR is generated in the authoring application preferences of these end users. It presents the reason for of COMPASS™, the author will publish the report so that genetic testing, information about the gene(s), variant(s) Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 5 of 9 Fig. 3 The published patient report. Overview. Shows the report overview which includes demographic information, a description of the reason for testing, primary and additional findings, and contact information and associated diagnoses, prognosis, information for was presented to providers and input from the providers other family members, a glossary of genetic terminology, was used to modify the draft [12]. The provider report and resources to learn more such as disorder-specific includes standard information such as demographics, support groups and sites designed for lay persons like clinical data and testing information. The report high- Genetics Home Reference (Fig. 4). To promote engage- lights the indication for genetic testing, key findings, pri- ment and empowerment, information is also provided mary and secondary diagnoses and variant information, about possible clinical trials relevant to the disorder any genes that were found that require more research (clinicaltrials.gov) and services such as GenomeConnect before conclusions can be drawn, a glossary of genetic that support patient/family entered data to facilitate net- terminology, and provider focused resources to learn working and research. Full details about the report are more such as GeneReviews and Online Mendelian In- published elsewhere [11]. heritance in Man. The providers requested access to the The provider-facing report (Fig. 5) is also made avail- PGR so that they are aware of information provided to able at the point of care to primary care and genetic spe- the patients and their families. Access to the PGR is cialty providers who have access to COMPASS™ in Epic. through a link within the provider report. COMPASS™ is embedded in the provider’s workflow in One additional enhancement that was added after the Epic in the visit navigator section and is designed to ei- final reports were developed was the inclusion of info- ther be a manually or automatically launched depending buttons. Infobuttons are context-sensitive links embed- on the clinic site’s needs. This report does not replace ded within information systems which allow easy the actual laboratory report which is still scanned into retrieval of relevant information. They use contextual in- the patient record (and is accessible through a link from formation about the patient, user, clinical setting, and the provider-facing report). It is meant to supplement EHR task to anticipate providers’ information needs and the laboratory report to assist providers as they interpret provide links to online clinical resources that may meet the results and use them to guide care. This draft report these information needs. While primarily directed to Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 6 of 9 Fig. 4 The published patient report: Primary finding detail. Shows the first portion of the report on the primary finding, in this case a pathogenic variant in the MECP2 gene Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 7 of 9 Fig. 5 The published provider report: Summary. This shows the provider view of the summary of findings. As with the patient report, the provider can drill down for additional information on both the primary and additional findings for the patient. In addition, the provider can link to the SimulConsult® patient summary, access the report issued by the testing laboratory and can view (but not edit) the full content of the patient report providers, there are emerging use cases that include this open source standard and its supporting resources infobuttons in patient-facing applications. Our enhance- allowed incorporation of the functionality into the PGR ment took advantage of the emergence of OpenInfobut- with minimal effort from the development team using ton, an open source suite of Web services that enable the SaaS functionality in COMPASS. This was a high infobutton capabilities within EHR systems [15]. Use of value addition to the report given that the Clinical Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 8 of 9 Genome resource, which aggregates genetic and gen- that GPRs will be issued when the participant is con- omic information from a variety of sources, is accessible tacted. The large number of reports will allow robust through OpenInfobutton [16]. evaluation and improvement. In addition, a pharmacoge- nomic (PGx) PGR has been developed [19]. Because Discussion PGx results are quite different from causal Mendelian The formative work that preceded development of the genetic variants, the PGx PGR was developed using the tool identified the need for such a tool as well as the same end user engagement approach as the PGR de- elements that were needed for inclusion. The proto- scribed here and elsewhere. Additional use cases are be- type underwent beta testing in the EHR and patient ing explored. portal environments to identify and fix any technical Demonstration of compatibility with other tools such problems that interfere with the tool’s access, per- as the SGPA enhances the utility and generalizability of formance and usability. The tool was deployed as part the PGR tool. This implementation required a significant of a prospective randomized comparative effectiveness amount of effort from personnel at both sites as it re- trial [13]. This trial used participants in a clinical re- quired development of novel interfaces and customized search project that is using whole genome sequencing local solutions. Standards-based approaches would be to determine the cause of undiagnosed intellectual desirable to allow more rapid enhancements such as was disability. The trial was recently completed and con- used to incorporate OpenInfobutton functionality, which firmed that the report improved satisfaction, engage- required very little developer time and no local ment and enhanced communication compared to customization. Recently, SMART on FHIR has been tou- standard of care for those participants where a causal ted as a new approach that will enable “plug-in apps” to variant was found. Qualitative evaluation identified run natively inside any compliant EHR [20]. The some technical issues that initially limited accessibility HL7-developed FHIR standards are designed to support of the PGR. When the PGR was launched for a pa- retrieval of data services needed to drive applications, tient, the message generated by the MyGeisinger por- while the SMART platform is designed to launch from tal was a default generic message that notified the the EHR and facilitate the launching of apps, all within recipient that a survey was available. This was confus- extant clinical workflows. Combined with OAuth2 ing and led many patients to delete the message with- authorization, the SMART on FHIR platform provides a out accessing the report. A customized message has standards-based technology stack for app development. been developed to solve this issue. Another problem Geisinger is currently exploring usage of SMART on was when the intended recipient was a caregiver of FHIR capabilities in platforms and applications including the patient, the caregiver had to have proxy access to COMPASS™. Most relevant to the PGR, SMART on the MyGeisinger portal to receive the message and FHIR Genomics is emerging as a genomic-specific HL7 launch the PGR. Proxy access is now addressed dur- approved standard [21]. The feasibility of the prototype ing the clinical encounter where the results are re- was demonstrated through incorporation in three repre- ported and caregivers are given instructions to create sentative applications based on the SMART platform. proxy access. Recently, SMART on FHIR Genomics has been selected While the tool has been developed as part of a larger for use in the All of Us program (previously the Preci- project involving whole genome sequencing, the tool sion Medicine Initiative) through the Sync for Genes could be used to supplement traditional genetic test re- project [22]. Incorporating the functionality of SMART ports as well. The tool also has the potential to support on FHIR Genomics into the PGR is being explored. As large scale sequencing projects with return of results. adoption of these standards increases, so will the poten- Geisinger is partnering with the Regeneron Genetics tial scalability and portability of current and future web Center to perform exome sequencing on as many as applications such as the PGR. 250,000 Geisinger patients over the next 5 years [17]. Over 90,000 exomes are already completed and analysis to identify pathogenic variants in genes that have been Conclusions determined to have clinical actionability (such as BRCA In conclusion, we believe that to realize the promise of 1 and 2, LDLR, etc.) [18]. The PGR is being used to sup- genomic and ultimately precision medicine requires in- port the clinical reporting of actionable genomic find- creased engagement with patients and their caregivers. ings. As of December 2017, over 500 clinical reports The PGR represents the first effort to develop and im- have been issued. PGRs have been issued for 61 partici- plement a tool to engage patients and their providers pants and their providers with the remaining 500 priori- within the EHR ecosystem. It addresses currently identi- tized for release over the next 3 months. Going forward, fied barriers and implements all the elements of the the clinical reporting process has been modified such ONCHIT Self-care and Community domains [8]. Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 9 of 9 Abbreviations 3. Hoffman MA, Williams MS. Electronic medical records and personalized API: Application Program Interface; EHR: Electronic Health Record; FHIR: Fast medicine. Hum Genet. 2011;130(1):33–9. Healthcare Interoperable Resources; HTML: Hypertext Markup Language; MS 4. Kullo IJ, Jarvik GP, Manolio TA, et al. Leveraging the electronic health record SQL: Microsoft Structured Query Language; ONCHIT: Office of the National to implement genomic medicine. Genet Med. 2013;15(4):270–1. Coordinator of Health Information Technology; PGR: Patient-facing Genomic 5. Manolio TA, Chisolm RL, Ozenberger B, et al. Implementing genomic test Report; PGx: Pharmacogenomics; SGPA: Simulconsult® Genome medicine in the clinic: the future is Here. Genet Med. 2013;15(4):258–67. Phenome Analyzer; SMART: Substitutable Medical Apps, Reusable 6. Lobach D, Sanders GD, Bright TJ, et al. Enabling health care decision making Technology; SOAP: Simple Object Access Protocol; SQL: Structured Query through clinical decision support and knowledge management. Evid Rep Language; XML: eXtensible Markup Language Technol Assess. 2012;203:1–84. 7. Delbanco T, Walker J, Bell SK, et al. Inviting patients to read their doctors’ notes: a quasi-experimental study and a look ahead. Ann Intern Med. 2012; Acknowledgements 157:461–70. Thanks to all the patient, their families, and providers that participated in the 8. Ahern DK, Woods SS, Lightowler MC, et al. Promise of potential for patient- development of the reports. We also acknowledge the significant facing technologies to enable meaningful use. Am J Prev Med. 2011;40(5 contributions of the SimulConsult® team, particularly Lynn Feldman and Suppl 2):S162–72. Michael Segal. 9. National Academies of Sciences, Engineering, and Medicine. Improving Diagnosis in Health Care. Washington, DC: The National Academies Press; Funding 2015. https://doi.org/10.17226/21794. Accessed 16 Dec 2017 Funded through Patient Centered Outcomes Research Institute; PCORI 10. Haga SB, Mills R, Pollak KI, et al. Developing patient-friendly genetic and Contract Number: CD-1304-6987. Project title: Enhancing Genomic Labora- genomic test reports: formats to promote patient engagement and tory Reports to Enhance Communication and Empower Patients. The funder understanding. Genome Med. 2014;6(7):58. had no role in the design of the study, collection, analysis and interpretation 11. Stuckey H, Williams JL, Fan AL, et al. Enhancing genomic laboratory reports of data, or writing the manuscript. from the patients’ view: a qualitative analysis. Am J Med Genet A. 2015; 167A(10):2238–43. Availability of data and materials 12. Williams JL, Rahm AK, Stuckey H, et al. Enhancing genomic laboratory The COMPASS™ software and interface was constructed using open source reports: a qualitative analysis of provider review. Am J Med Genet A. 2016; standards as described in the materials and methods. Questions about the 170A(5):1134–41. technical details or implementation can be directed to the corresponding 13. Williams JL, Rahm AK, Zallen DT, et al. Impact of a patient-facing enhanced author. The Simulconsult® Genome Phenome Analyzer (SGPA) is a genomic results report to improve understanding, engagement, and proprietary software tool. As a result, the PGR that is integrated with the communication J Genet Couns 2017. doi: https://doi.org/10.1007/s10897- SGPA is not available for distribution. Inquiries about SGPA can be directed 017-0176-6. [Epub ahead of print]. to Dr. Michael Segal, or Lynn Feldman, by e-mail contactYYYY@simulconsult.- 14. http://simulconsult.com/genome/index.html. Accessed 16 Dec 2017. comwhere YYYY is the year. In the subject line, state the intended recipient 15. http://www.openinfobutton.org/. Accessed 16 Dec 2017. (person or role). Additional information on this software is available at 16. Heale BSE, Overby CL, Del Fiol G, et al. Integrating genomic resources with www.simulconsult.com. electronic health records using the HL7 Infobutton standard. Appl Clin Inform. 2016;3:817–31. Authors’ contributions 17. Carey DJ, Fetterolf SN, Davis FD, et al. The Geisinger MyCode community All authors participated in the drafting, review and approval of the health initiative: an electronic health record-linked biobank for precision manuscript. MSW and JLW conceived the project and led the design efforts. medicine research. Genet Med. 2016;18(9):906–13. GJM contributed to the conceptual design of the tool and supervised the 18. Dewey FE, Murray MF, Overton JD, et al. Distribution and clinical impact of design team. JB, MSK and VRL created the tool, led the testing and functional variants in 50,726 whole-exome sequences from the DiscovEHR implementation and participated in the design decisions. All authors have study. Science. 2016;354(6319) reviewed and approved the final manuscript. 19. Jones LK, Rahm AK, Gionfriddo MR, et al. Developing pharmacogenomic reports: insights from patients and clinicians. Clin Transl Sci. 2018;11(3):289–95. Ethics approval and consent to participate 20. Something New and Powerful: SMART on FHIR. 2015. http://smarthealthit. Participants involved in the development of the tool provided written org/smart-on-fhir/ Accessed 16 Dec 2017. consent to participate. This study was approved by the Geisinger Institutional 21. Alterovitz G, Warner J, Zhang P, et al. SMART on FHIR genomics: facilitating Review Board Registration # 00008345. standardized clinico-genomic apps. J Am Med Inform Assoc. 2015;22(6): 1173–8. Competing interests 22. https://www.healthcare-informatics.com/article/interoperability/sync-genes- The authors declare that they have no competing interests. complement-sync-science. Accessed 16 Dec 2017. Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Author details Genomic Medicine Institute, Geisinger, 100 North Academy Avenue, Danville, PA, USA. Center for Pharmacy Innovation and Outcomes, Geisinger, Danville, PA, USA. Institute for Advanced Application, Geisinger, Danville, PA, USA. Received: 2 May 2017 Accepted: 22 May 2018 References 1. Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med. 2015;372(9):793–5. 2. Rare Disease Information. National Organization for Rare Disorders http:// rarediseases.org/for-patients-and-families/information-resources/rare-disease- information/ Accessed 16 Dec 2017. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png BMC Medical Informatics and Decision Making Springer Journals

Implementation of a patient-facing genomic test report in the electronic health record using a web-application interface

Free
9 pages
Loading next page...
 
/lp/springer_journal/implementation-of-a-patient-facing-genomic-test-report-in-the-TW57QAFDjJ
Publisher
BioMed Central
Copyright
Copyright © 2018 by The Author(s).
Subject
Medicine & Public Health; Health Informatics; Information Systems and Communication Service; Management of Computing and Information Systems
eISSN
1472-6947
D.O.I.
10.1186/s12911-018-0614-x
Publisher site
See Article on Publisher Site

Abstract

Background: Genomic medicine is emerging into clinical care. Communication of genetic laboratory results to patients and providers is hampered by the complex technical nature of the laboratory reports. This can lead to confusion and misinterpretation of the results resulting in inappropriate care. Patients usually do not receive a copy of the report leading to further opportunities for miscommunication. To address these problems, interpretive reports were created using input from the intended end users, patients and providers. This paper describes the technical development and deployment of the first patient-facing genomic test report (PGR) within an electronic health record (EHR) ecosystem using a locally developed standards-based web-application interface. Methods: A patient-facing genomic test report with a companion provider report was configured for implementation within the EHR using a locally developed software platform, COMPASS™.COMPASS™ is designed to manage secure data exchange, as well as patient and provider access to patient reported data capture and clinical display tools. COMPASS™ is built using a Software as a Service (SaaS) approach which exposes an API that apps can interact with. Results: An authoring tool was developed that allowed creation of patient-specific PGRs and the accompanying provider reports. These were converted to a format that allowed them to be presented in the patient portal and EHR respectively using the existing COMPASS™ interface thus allowing patients, caregivers and providers access to individual reports designed for the intended end user. Conclusions: The PGR as developed was shown to enhance patient and provider communication around genomic results. It is built on current standards but is designed to support integration with other tools and be compatible with emerging opportunities such as SMART on FHIR. This approach could be used to support genomic return of results as the tool is scalable and generalizable. Keywords: Genomics, Medical informatics applications, Electronic health records, Patient-centered care, Patient access to records, Communication Background challenge for patients and their providers is having ready Precision Medicine with an emphasis on the incorporation access to the information that is necessary for appropri- of genomic information into the clinic has been identified ate management and coordination of care. When faced as a priority for research in the United States [1]. with a patient with a genetic condition they have not Genetic disorders, while individually rare, are collect- previously encountered, knowledge and resources are ively common. It is estimated that there are over 6800 extremely limited. This often puts patients and families rare and ultra-rare disorders, many of which are genetic in the position of attempting to become the ‘expert’ in affecting approximately 30 million Americans [2]. The the specific disease via an unguided internet search process which can lead to uncomfortable and sometimes adversarial encounters with the medical system. * Correspondence: mswilliams1@geisinger.edu One potential solution to lower the barriers experi- Genomic Medicine Institute, Geisinger, 100 North Academy Avenue, enced by providers caring for patients with genetic Danville, PA, USA Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 2 of 9 conditions is using fully functional electronic health rec- recently concluded and demonstrates qualitative im- ord systems (EHR) [3–5]. The capabilities provided by provement in communication and satisfaction. such EHRs, particularly through knowledge management The purpose of this paper is to present the technical systems and clinical decision support systems, have been aspects of development and deployment of the first demonstrated to significantly improve process outcomes, patient-facing genomic test report (PGR) within an EHR although the evidence of impact on health outcomes is ecosystem using a locally developed web-application less robust [6]. interface. While these techniques have mostly faced toward pro- viders, similar approaches could be used to provide in- Methods formation directly to patients and families. A recent A PGR was developed using a mixed-methods formative study conducted at 3 healthcare organizations, including approach which has been previously reported [11]. The Geisinger, studied the impact of opening patient access PGR includes content that was considered important by to their provider notes through a secure patient portal patients and their families. The content is vetted for (OpenNotes) [7]. Using a quasi-experimental trial de- readability through a review process that includes con- sign, several significant differences were noted when tent experts, non-genetic providers and patients and is comparing the pre- and post-intervention surveys in- formally tested to meet accepted literacy standards. The cluding patients reporting that OpenNotes, “…helped PGR was developed using paper mock-ups. Once the them feel more in control of their care” and “…increased final format was chosen, this was converted to an elec- medication adherence:”. A recent article noted that tronic format that could be displayed through an inter- patient-facing applications could enable meaningful use face such as a patient portal, but could also be printed objectives promoted by the Office of the National Co- to provide a hard copy. A similar process was used to ordinator of Health Information Technology (ONCHIT) create a provider version of the PGR that was designed [8]. This approach is also endorsed by the recent report to complement the PGR [12]. These two reports were from the National Academy of Medicine (formerly the then submitted to the COMPASS™ development team Institute of Medicine), Improving Diagnosis in Health for implementation. Care [9] which highlights the importance of patient COMPASS™ is a software platform developed at Gei- empowerment. singer intended to work as an add-on to EHRs. Its main One area that has received less emphasis is the role of purpose is to manage secure data exchange, as well as the laboratory report in communicating information to patient and provider access to patient reported data cap- patients. Haga et al. [10] propose four revisions to ture and clinical display tools utilized within Geisinger. current genomic test reports that could improve accessi- It is intended to improve patient engagement in their bility for patients. These include, “…1) inclusion of an care and enhance patient-provider communication. interpretive summary section, 2) a summary letter to ac- COMPASS™ is built using the ASP.Net framework company the lab report, 3) development of a patient user where the presentation layer (User Interface) imple- guide to be provided with the report, and 4) a com- ments web development languages and libraries includ- pletely revised patient-friendly report.” In genomic medi- ing HTML, CSS, JavaScript, JQuery, AngularJS, and cine, direct-to-consumer companies have emerged that Angular2. The back-end (Code Behind, Logic, Data In- provide results of tests to consumers without the inter- teractions) is coded using C# and utilizes SQL Data mediary of a health care professional. While many of the Adapters, SOAP Web Services, WebAPIs, and Teradata results are non-health related (e.g. ancestry) some com- Data Adapters to facilitate data transfers. The COM- panies are returning predictive information that could PASS™ data model utilizes MS SQL Server but it is not have an impact on an individual’s health. The positive limited to only utilizing SQL Server for interacting with and negative outcomes of providing this information dir- data as it can consume data from web services or any ectly to consumers have not been rigorously studied. standard database platform through its Software as a Our group has previously reported on the design and Service (SaaS) approach. Through this, users can either testing of linked patient- and provider-facing genomic pass data to the COMPASS™API for use or COMPASS™ test reports [11, 12]. These reports are not meant to re- can access common database platforms and consume/ place the laboratory report, but to provide additional in- display appropriate data. It is designed so that any con- formation to guide interpretation and clarify clinical tent can be coded to be accessible on any other resource. recommendations that result from the testing. As such This allows content in COMPASS™ to be agnostic from they are compatible with the recommendation from its own server/resource perspective. Haga to create a “a patient-friendly report” [10]. A com- COMPASS™ is integrated with Geisinger’s primary parative effectiveness study [13] that compares the use EHR (Epic) on several levels. The EHR interacts with of these reports to standard of care communication has COMPASS™ by launching an external web browser via a Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 3 of 9 specific encrypted URL. The current implementation patient specific reports. Through COMPASS™, the pa- within Epic is a visit navigator link which contains re- tient report template interfaces with various data sources quired encrypted parameters for authentication. COM- in addition to the database storing the information gen- PASS™ is fed by real-time data feeds from the EHR to erated from the generic gene template (Fig. 2), inherit- correctly determine project inclusion/exclusion and dis- ance pattern template, and glossary term template. One play correct and current data to users. Currently, the specific tool, the SimulConsult® Genome-Phenome real time feeds that COMPASS™ utilizes are custom and Analyzer (SGPA) [14] has been used to provide informa- standard data extractions that interface with Epic. COM- tion to the report. A local API is called when a user fin- PASS™ is also integrated with patient portals (currently ishes creating the content for the custom report in the MyGeisinger which is a MyChart implementation) so authoring environment. The author then opens the Epic patients can use the application when off-site from the record for the patient and launches the SGPA from the health system network. As with the EHR, COMPASS™ encounter navigator. In the SGPA, patient information is interacts with the portal by launching an external web combined with the genomic sequence information and a browser via an encrypted URL. gene associated with the clinical features along with a An authoring application was created within COM- variant in that gene are selected by the author. This PASS™ to assist geneticists and genetic counselors with ‘closes the loop’ by associating the gene and variant with building coordinated and complementary patient and the specific patient. The author then clicks the “Send family-facing and provider-facing reports of the patient’s XML Report” button in the SGPA which sends Simul- test results. The authoring application includes areas to Consult® content back to the authoring environment. A create and store generic genetic content for the reports custom parser developed to support the project, in com- that is not readily available from other sources. The bination with the SimulConsult® Web API loads the data authoring environment contains reusable content fields into the “PatientReports” table in the Genomics data- (including inheritance patterns, glossary terms and gene base. The final patient-specific report is available for re- templates that create gene-specific content that can be view and publication at which time it appears in the used for all reports that return results related to the MyGeisinger patient portal, and is accessible to clini- gene) that can be selected to populate patient and pro- cians in the patient’s encounter navigator through a local vider reports. The environment tracks versioning of the Web Apps URL. One specific example of SGPA content information created within the authoring application— that was highly valued by both patients and providers an essential element given the rapidly changing nature was the prognosis table. This prognosis table, which of knowledge in genomics that can have implications for summarizes medical issues associated with a condition liability. The ecosystem of the PGR report is presented the likelihood that the issue will develop and when in in Fig. 1. the life of the patient they are likely to appear, is auto- A patient report template is used by the geneticist/ matically generated by the SGPA [14]. The manual cre- counselor to pull information together to create the ation of such a table would require a tremendous Fig. 1 COMPASS™ Data Flow. A diagram of the data flows for COMPASS™. Note that the EHR data flow is in the process of moving to SMART on FHIR which should increase the generalizability of the tool in different EHR ecosystems Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 4 of 9 Fig. 2 The generic gene template. An excerpt of the MECP2 gene template from the COMPASS™ authoring tool investment of time and expertise. While not required it is available to patients and their authorized representa- for the authoring process, use of the SGPA improves tives (usually family members). Patients and their fam- the workflow and reduces the time needed to author ilies can access the report electronically through a link a report. Upon selection and verification of the con- out to COMPASS™ from the patient’s MyGeisinger por- tent, the report content is published from the author- tal or prior to a visit in the clinic during the nurse room- ing application and made available to patients and ing process in Epic (Fig. 3). The PGR is also designed so their families and providers electronically in that a hard copy can be printed—a requirement that was COMPASS™. requested by participants in the formative study. The re- port is written and laid out in a manner based on input Results from the interviews and focus groups that reflects the After the PGR is generated in the authoring application preferences of these end users. It presents the reason for of COMPASS™, the author will publish the report so that genetic testing, information about the gene(s), variant(s) Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 5 of 9 Fig. 3 The published patient report. Overview. Shows the report overview which includes demographic information, a description of the reason for testing, primary and additional findings, and contact information and associated diagnoses, prognosis, information for was presented to providers and input from the providers other family members, a glossary of genetic terminology, was used to modify the draft [12]. The provider report and resources to learn more such as disorder-specific includes standard information such as demographics, support groups and sites designed for lay persons like clinical data and testing information. The report high- Genetics Home Reference (Fig. 4). To promote engage- lights the indication for genetic testing, key findings, pri- ment and empowerment, information is also provided mary and secondary diagnoses and variant information, about possible clinical trials relevant to the disorder any genes that were found that require more research (clinicaltrials.gov) and services such as GenomeConnect before conclusions can be drawn, a glossary of genetic that support patient/family entered data to facilitate net- terminology, and provider focused resources to learn working and research. Full details about the report are more such as GeneReviews and Online Mendelian In- published elsewhere [11]. heritance in Man. The providers requested access to the The provider-facing report (Fig. 5) is also made avail- PGR so that they are aware of information provided to able at the point of care to primary care and genetic spe- the patients and their families. Access to the PGR is cialty providers who have access to COMPASS™ in Epic. through a link within the provider report. COMPASS™ is embedded in the provider’s workflow in One additional enhancement that was added after the Epic in the visit navigator section and is designed to ei- final reports were developed was the inclusion of info- ther be a manually or automatically launched depending buttons. Infobuttons are context-sensitive links embed- on the clinic site’s needs. This report does not replace ded within information systems which allow easy the actual laboratory report which is still scanned into retrieval of relevant information. They use contextual in- the patient record (and is accessible through a link from formation about the patient, user, clinical setting, and the provider-facing report). It is meant to supplement EHR task to anticipate providers’ information needs and the laboratory report to assist providers as they interpret provide links to online clinical resources that may meet the results and use them to guide care. This draft report these information needs. While primarily directed to Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 6 of 9 Fig. 4 The published patient report: Primary finding detail. Shows the first portion of the report on the primary finding, in this case a pathogenic variant in the MECP2 gene Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 7 of 9 Fig. 5 The published provider report: Summary. This shows the provider view of the summary of findings. As with the patient report, the provider can drill down for additional information on both the primary and additional findings for the patient. In addition, the provider can link to the SimulConsult® patient summary, access the report issued by the testing laboratory and can view (but not edit) the full content of the patient report providers, there are emerging use cases that include this open source standard and its supporting resources infobuttons in patient-facing applications. Our enhance- allowed incorporation of the functionality into the PGR ment took advantage of the emergence of OpenInfobut- with minimal effort from the development team using ton, an open source suite of Web services that enable the SaaS functionality in COMPASS. This was a high infobutton capabilities within EHR systems [15]. Use of value addition to the report given that the Clinical Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 8 of 9 Genome resource, which aggregates genetic and gen- that GPRs will be issued when the participant is con- omic information from a variety of sources, is accessible tacted. The large number of reports will allow robust through OpenInfobutton [16]. evaluation and improvement. In addition, a pharmacoge- nomic (PGx) PGR has been developed [19]. Because Discussion PGx results are quite different from causal Mendelian The formative work that preceded development of the genetic variants, the PGx PGR was developed using the tool identified the need for such a tool as well as the same end user engagement approach as the PGR de- elements that were needed for inclusion. The proto- scribed here and elsewhere. Additional use cases are be- type underwent beta testing in the EHR and patient ing explored. portal environments to identify and fix any technical Demonstration of compatibility with other tools such problems that interfere with the tool’s access, per- as the SGPA enhances the utility and generalizability of formance and usability. The tool was deployed as part the PGR tool. This implementation required a significant of a prospective randomized comparative effectiveness amount of effort from personnel at both sites as it re- trial [13]. This trial used participants in a clinical re- quired development of novel interfaces and customized search project that is using whole genome sequencing local solutions. Standards-based approaches would be to determine the cause of undiagnosed intellectual desirable to allow more rapid enhancements such as was disability. The trial was recently completed and con- used to incorporate OpenInfobutton functionality, which firmed that the report improved satisfaction, engage- required very little developer time and no local ment and enhanced communication compared to customization. Recently, SMART on FHIR has been tou- standard of care for those participants where a causal ted as a new approach that will enable “plug-in apps” to variant was found. Qualitative evaluation identified run natively inside any compliant EHR [20]. The some technical issues that initially limited accessibility HL7-developed FHIR standards are designed to support of the PGR. When the PGR was launched for a pa- retrieval of data services needed to drive applications, tient, the message generated by the MyGeisinger por- while the SMART platform is designed to launch from tal was a default generic message that notified the the EHR and facilitate the launching of apps, all within recipient that a survey was available. This was confus- extant clinical workflows. Combined with OAuth2 ing and led many patients to delete the message with- authorization, the SMART on FHIR platform provides a out accessing the report. A customized message has standards-based technology stack for app development. been developed to solve this issue. Another problem Geisinger is currently exploring usage of SMART on was when the intended recipient was a caregiver of FHIR capabilities in platforms and applications including the patient, the caregiver had to have proxy access to COMPASS™. Most relevant to the PGR, SMART on the MyGeisinger portal to receive the message and FHIR Genomics is emerging as a genomic-specific HL7 launch the PGR. Proxy access is now addressed dur- approved standard [21]. The feasibility of the prototype ing the clinical encounter where the results are re- was demonstrated through incorporation in three repre- ported and caregivers are given instructions to create sentative applications based on the SMART platform. proxy access. Recently, SMART on FHIR Genomics has been selected While the tool has been developed as part of a larger for use in the All of Us program (previously the Preci- project involving whole genome sequencing, the tool sion Medicine Initiative) through the Sync for Genes could be used to supplement traditional genetic test re- project [22]. Incorporating the functionality of SMART ports as well. The tool also has the potential to support on FHIR Genomics into the PGR is being explored. As large scale sequencing projects with return of results. adoption of these standards increases, so will the poten- Geisinger is partnering with the Regeneron Genetics tial scalability and portability of current and future web Center to perform exome sequencing on as many as applications such as the PGR. 250,000 Geisinger patients over the next 5 years [17]. Over 90,000 exomes are already completed and analysis to identify pathogenic variants in genes that have been Conclusions determined to have clinical actionability (such as BRCA In conclusion, we believe that to realize the promise of 1 and 2, LDLR, etc.) [18]. The PGR is being used to sup- genomic and ultimately precision medicine requires in- port the clinical reporting of actionable genomic find- creased engagement with patients and their caregivers. ings. As of December 2017, over 500 clinical reports The PGR represents the first effort to develop and im- have been issued. PGRs have been issued for 61 partici- plement a tool to engage patients and their providers pants and their providers with the remaining 500 priori- within the EHR ecosystem. It addresses currently identi- tized for release over the next 3 months. Going forward, fied barriers and implements all the elements of the the clinical reporting process has been modified such ONCHIT Self-care and Community domains [8]. Williams et al. BMC Medical Informatics and Decision Making (2018) 18:32 Page 9 of 9 Abbreviations 3. Hoffman MA, Williams MS. Electronic medical records and personalized API: Application Program Interface; EHR: Electronic Health Record; FHIR: Fast medicine. Hum Genet. 2011;130(1):33–9. Healthcare Interoperable Resources; HTML: Hypertext Markup Language; MS 4. Kullo IJ, Jarvik GP, Manolio TA, et al. Leveraging the electronic health record SQL: Microsoft Structured Query Language; ONCHIT: Office of the National to implement genomic medicine. Genet Med. 2013;15(4):270–1. Coordinator of Health Information Technology; PGR: Patient-facing Genomic 5. Manolio TA, Chisolm RL, Ozenberger B, et al. Implementing genomic test Report; PGx: Pharmacogenomics; SGPA: Simulconsult® Genome medicine in the clinic: the future is Here. Genet Med. 2013;15(4):258–67. Phenome Analyzer; SMART: Substitutable Medical Apps, Reusable 6. Lobach D, Sanders GD, Bright TJ, et al. Enabling health care decision making Technology; SOAP: Simple Object Access Protocol; SQL: Structured Query through clinical decision support and knowledge management. Evid Rep Language; XML: eXtensible Markup Language Technol Assess. 2012;203:1–84. 7. Delbanco T, Walker J, Bell SK, et al. Inviting patients to read their doctors’ notes: a quasi-experimental study and a look ahead. Ann Intern Med. 2012; Acknowledgements 157:461–70. Thanks to all the patient, their families, and providers that participated in the 8. Ahern DK, Woods SS, Lightowler MC, et al. Promise of potential for patient- development of the reports. We also acknowledge the significant facing technologies to enable meaningful use. Am J Prev Med. 2011;40(5 contributions of the SimulConsult® team, particularly Lynn Feldman and Suppl 2):S162–72. Michael Segal. 9. National Academies of Sciences, Engineering, and Medicine. Improving Diagnosis in Health Care. Washington, DC: The National Academies Press; Funding 2015. https://doi.org/10.17226/21794. Accessed 16 Dec 2017 Funded through Patient Centered Outcomes Research Institute; PCORI 10. Haga SB, Mills R, Pollak KI, et al. Developing patient-friendly genetic and Contract Number: CD-1304-6987. Project title: Enhancing Genomic Labora- genomic test reports: formats to promote patient engagement and tory Reports to Enhance Communication and Empower Patients. The funder understanding. Genome Med. 2014;6(7):58. had no role in the design of the study, collection, analysis and interpretation 11. Stuckey H, Williams JL, Fan AL, et al. Enhancing genomic laboratory reports of data, or writing the manuscript. from the patients’ view: a qualitative analysis. Am J Med Genet A. 2015; 167A(10):2238–43. Availability of data and materials 12. Williams JL, Rahm AK, Stuckey H, et al. Enhancing genomic laboratory The COMPASS™ software and interface was constructed using open source reports: a qualitative analysis of provider review. Am J Med Genet A. 2016; standards as described in the materials and methods. Questions about the 170A(5):1134–41. technical details or implementation can be directed to the corresponding 13. Williams JL, Rahm AK, Zallen DT, et al. Impact of a patient-facing enhanced author. The Simulconsult® Genome Phenome Analyzer (SGPA) is a genomic results report to improve understanding, engagement, and proprietary software tool. As a result, the PGR that is integrated with the communication J Genet Couns 2017. doi: https://doi.org/10.1007/s10897- SGPA is not available for distribution. Inquiries about SGPA can be directed 017-0176-6. [Epub ahead of print]. to Dr. Michael Segal, or Lynn Feldman, by e-mail contactYYYY@simulconsult.- 14. http://simulconsult.com/genome/index.html. Accessed 16 Dec 2017. comwhere YYYY is the year. In the subject line, state the intended recipient 15. http://www.openinfobutton.org/. Accessed 16 Dec 2017. (person or role). Additional information on this software is available at 16. Heale BSE, Overby CL, Del Fiol G, et al. Integrating genomic resources with www.simulconsult.com. electronic health records using the HL7 Infobutton standard. Appl Clin Inform. 2016;3:817–31. Authors’ contributions 17. Carey DJ, Fetterolf SN, Davis FD, et al. The Geisinger MyCode community All authors participated in the drafting, review and approval of the health initiative: an electronic health record-linked biobank for precision manuscript. MSW and JLW conceived the project and led the design efforts. medicine research. Genet Med. 2016;18(9):906–13. GJM contributed to the conceptual design of the tool and supervised the 18. Dewey FE, Murray MF, Overton JD, et al. Distribution and clinical impact of design team. JB, MSK and VRL created the tool, led the testing and functional variants in 50,726 whole-exome sequences from the DiscovEHR implementation and participated in the design decisions. All authors have study. Science. 2016;354(6319) reviewed and approved the final manuscript. 19. Jones LK, Rahm AK, Gionfriddo MR, et al. Developing pharmacogenomic reports: insights from patients and clinicians. Clin Transl Sci. 2018;11(3):289–95. Ethics approval and consent to participate 20. Something New and Powerful: SMART on FHIR. 2015. http://smarthealthit. Participants involved in the development of the tool provided written org/smart-on-fhir/ Accessed 16 Dec 2017. consent to participate. This study was approved by the Geisinger Institutional 21. Alterovitz G, Warner J, Zhang P, et al. SMART on FHIR genomics: facilitating Review Board Registration # 00008345. standardized clinico-genomic apps. J Am Med Inform Assoc. 2015;22(6): 1173–8. Competing interests 22. https://www.healthcare-informatics.com/article/interoperability/sync-genes- The authors declare that they have no competing interests. complement-sync-science. Accessed 16 Dec 2017. Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Author details Genomic Medicine Institute, Geisinger, 100 North Academy Avenue, Danville, PA, USA. Center for Pharmacy Innovation and Outcomes, Geisinger, Danville, PA, USA. Institute for Advanced Application, Geisinger, Danville, PA, USA. Received: 2 May 2017 Accepted: 22 May 2018 References 1. Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med. 2015;372(9):793–5. 2. Rare Disease Information. National Organization for Rare Disorders http:// rarediseases.org/for-patients-and-families/information-resources/rare-disease- information/ Accessed 16 Dec 2017.

Journal

BMC Medical Informatics and Decision MakingSpringer Journals

Published: May 30, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off