Access the full text.
Sign up today, get DeepDyve free for 14 days.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova (2019)
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
H. Chase, L. Mitrani, Gabriel Lu, Dominick Fulgieri (2017)
Early recognition of multiple sclerosis using natural language processing of the electronic health recordBMC Medical Informatics and Decision Making, 17
Label studio
Theresa Koleck, C. Dreisbach, P. Bourne, S. Bakken (2019)
Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic reviewJournal of the American Medical Informatics Association, 26
Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAETM)
C. Cleeland (2007)
Symptom burden: multiple symptoms and their impact as patient-reported outcomes.Journal of the National Cancer Institute. Monographs, 37
K. Kehl, Wenxin Xu, E. Lepisto, Haitham Elmarakeby, M. Hassett, E. Allen, B. Johnson, D. Schrag (2020)
Natural Language Processing to Ascertain Cancer Outcomes From Medical Oncologist Notes.JCO clinical cancer informatics, 4
L. Bubis, L. Davis, A. Mahar, L. Barbera, Qing Li, L. Moody, P. Karanicolas, R. Sutradhar, N. Coburn (2018)
Symptom Burden in the First Year After Cancer Diagnosis: An Analysis of Patient-Reported Outcomes.Journal of clinical oncology : official journal of the American Society of Clinical Oncology, 36 11
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, Veselin Stoyanov (2019)
Unsupervised Cross-lingual Representation Learning at Scale
(2017)
Attention is all you need, 31st Conference on Neural Information Processing Systems (NIPS 2017)
Huaxiu Tang, I. Solti, E. Kirkendall, Haijun Zhai, T. Lingren, J. Meller, Yizhao Ni (2017)
Leveraging Food and Drug Administration Adverse Event Reports for the Automated Monitoring of Electronic Health Records in a Pediatric HospitalBiomedical Informatics Insights, 9
B. Rooij, N. Ezendam, F. Mols, P. Vissers, M. Thong, C. Vlooswijk, S. Oerlemans, O. Husson, N. Horevoorts, L. Poll-Franse (2018)
Cancer survivors not participating in observational patient-reported outcome studies have a lower survival compared to participants: the population-based PROFILES registryQuality of Life Research, 27
G. Weissman, M. Harhay, Ricardo Lugo, B. Fuchs, S. Halpern, M. Mikkelsen (2016)
Natural Language Processing to Assess Documentation of Features of Critical Illness in Discharge Documents of Acute Respiratory Distress Syndrome Survivors.Annals of the American Thoracic Society, 13 9
A. Fairchild, J. Tanksley, J. Tenenbaum, M. Palta, Julian Hong (2020)
Inter-rater reliability in toxicity identification: Limitations of current standards.International journal of radiation oncology, biology, physics
Ehtesham Iqbal, Robbie Mallah, D. Rhodes, Honghan Wu, A. Romero, N. Chang, O. Dzahini, Chandra Pandey, M. Broadbent, R. Stewart, R. Dobson, Zina Ibrahim (2017)
ADEPt, a semantically-enriched pipeline for extracting adverse drug events from free-text electronic health recordsPLoS ONE, 12
E. Basch, X. Jia, G. Heller, A. Barz, L. Sit, M. Fruscione, M. Appawu, A. Iasonos, T. Atkinson, S. Goldfarb, A. Culkin, M. Kris, D. Schrag (2009)
Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes.Journal of the National Cancer Institute, 101 23
B. Udelsman, E. Moseley, R. Sudore, N. Keating, C. Lindvall (2020)
Deep Natural Language Processing Identifies Variation in Care Preference Documentation.Journal of pain and symptom management
E. Basch, A. Deal, M. Kris, H. Scher, C. Hudis, P. Sabbatini, L. Rogak, A. Bennett, A. Dueck, T. Atkinson, J. Chou, Dorothy Dulko, L. Sit, A. Barz, P. Novotny, M. Fruscione, J. Sloan, D. Schrag (2016)
Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial.Journal of clinical oncology : official journal of the American Society of Clinical Oncology, 34 6
C. Lindvall, Chih-ying Deng, E. Moseley, Nicole Agaronnik, A. El-Jawahri, M. Paasche-Orlow, J. Lakin, A. Volandes, The Tulsky (2021)
Natural Language Processing to Identify Advance Care Planning Documentation in a Multisite Pragmatic Clinical Trial.Journal of pain and symptom management
K. Kehl, Haitham Elmarakeby, M. Nishino, E. Allen, E. Lepisto, M. Hassett, B. Johnson, D. Schrag (2019)
Assessment of Deep Natural Language Processing in Ascertaining Oncologic Outcomes From Radiology Reports.JAMA oncology
G. Savova, I. Danciu, Folami Alamudun, Timothy Miller, Chen Lin, D. Bitterman, G. Tourassi, J. Warner (2019)
Use of Natural Language Processing to Extract Clinical Cancer Phenotypes from Electronic Medical Records.Cancer research
Xiaoyan Wang, G. Hripcsak, M. Markatou, C. Friedman (2009)
Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: a feasibility study.Journal of the American Medical Informatics Association : JAMIA, 16 3
R. Byrd, S. Steinhubl, Jimeng Sun, S. Ebadollahi, W. Stewart (2014)
Automatic identification of heart failure diagnostic criteria, using text analysis of clinical notes from electronic health recordsInternational journal of medical informatics, 83 12
E. Sang, F. Meulder (2003)
Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition
Joshua Biber, D. Ose, J. Reese, A. Gardiner, J. Facelli, Joshua Spuhl, D. Brodke, Vivian Lee, R. Hess, H. Weeks (2018)
Patient reported outcomes – experiences with implementation in a University Health Care settingJournal of Patient-Reported Outcomes, 2
S. Tamang, Manali Patel, D. Blayney, J. Kuznetsov, Sam Finlayson, Yohan Vetteth, N. Shah (2015)
Detecting unplanned care from clinician notes in electronic health records.Journal of oncology practice, 11 3
Versione, Francesco Grossi (2020)
Common Terminology Criteria for Adverse EventsDefinitions
D. Bruner, L. Hanisch, B. Reeve, A. Trotti, D. Schrag, L. Sit, T. Mendoza, L. Minasian, A. O'Mara, A. Denicoff, J. Rowland, M. Montello, C. Geoghegan, A. Abernethy, S. Clauser, K. Castro, S. Mitchell, L. Burke, Ann Trentacosti, E. Basch (2011)
Stakeholder perspectives on implementing the National Cancer Institute’s patient-reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)Translational Behavioral Medicine, 1
Julian Hong, A. Fairchild, J. Tanksley, M. Palta, J. Tenenbaum (2020)
Natural language processing for abstraction of cancer treatment toxicities: accuracy versus human expertsJAMIA Open, 3
P. Kluetz, Diana Chingos, E. Basch, S. Mitchell (2016)
Patient-Reported Outcomes in Cancer Clinical Trials: Measuring Symptomatic Adverse Events With the National Cancer Institute's Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).American Society of Clinical Oncology educational book. American Society of Clinical Oncology. Annual Meeting, 35
T. Atkinson, Yuelin Li, C. Coffey, L. Sit, Mary Shaw, D. Lavene, A. Bennett, M. Fruscione, L. Rogak, J. Hay, M. Gönen, D. Schrag, E. Basch (2012)
Reliability of adverse symptom event reporting by cliniciansQuality of Life Research, 21
A. Yeung, S. Pugh, A. Klopp, K. Gil, L. Wenzel, S. Westin, D. Gaffney, W. Small, S. Thompson, D. Doncals, G. Cantuaria, B. Yaremko, A. Chang, V. Kundapur, D. Mohan, M. Haas, Y. Kim, C. Ferguson, S. Deshmukh, D. Bruner, L. Kachnic (2020)
Improvement in Patient-Reported Outcomes With Intensity-Modulated Radiotherapy (RT) Compared With Standard RT: A Report From the NRG Oncology RTOG 1203 Study.Journal of clinical oncology : official journal of the American Society of Clinical Oncology
E. Gamper, V. Nerich, M. Sztankay, Caroline Martini, J. Giesinger, L. Scarpa, S. Buxbaum, Martin Jeller, B. Holzner, I. Virgolini (2017)
Evaluation of Noncompletion Bias and Long-Term Adherence in a 10-Year Patient-Reported Outcome Monitoring Program in Clinical Routine.Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research, 20 4
Liana Yocavitch, A. Binder, A. Leader, Gregory Garber, A. Chapman, N. Handley (2019)
Challenges in implementing a mobile-based patient-reported outcome (PRO) tool for cancer patients.Journal of Clinical Oncology
Alex Chan, Isabel Chien, E. Moseley, Saad Salman, Sarah Bourland, D. Lamas, A. Walling, J. Tulsky, C. Lindvall (2018)
Deep learning algorithms to identify documentation of serious illness conversations during intensive care unit admissionsPalliative Medicine, 33
E. Basch, A. Deal, A. Dueck, H. Scher, M. Kris, C. Hudis, D. Schrag (2017)
Overall Survival Results of a Trial Assessing Patient-Reported Outcomes for Symptom Monitoring During Routine Cancer TreatmentJAMA, 318
Alexandra Pomares-Quimbaya, Markus Kreuzthaler, S. Schulz (2019)
Current approaches to identify sections within clinical narratives from electronic health records: a systematic reviewBMC Medical Research Methodology, 19
Wen-wai Yim, M. Yetisgen, W. Harris, S. Kwan (2016)
Natural Language Processing in Oncology: A Review.JAMA oncology, 2 6
A. Gayet-Ageron, T. Agoritsas, L. Schiesari, V. Kolly, T. Perneger (2011)
Barriers to Participation in a Patient Satisfaction Survey: Who Are We Missing?PLoS ONE, 6
PURPOSESymptoms are vital outcomes for cancer clinical trials, observational research, and population-level surveillance. Patient-reported outcomes (PROs) are valuable for monitoring symptoms, yet there are many challenges to collecting PROs at scale. We sought to develop, test, and externally validate a deep learning model to extract symptoms from unstructured clinical notes in the electronic health record.METHODSWe randomly selected 1,225 outpatient progress notes from among patients treated at the Dana-Farber Cancer Institute between January 2016 and December 2019 and used 1,125 notes as our training/validation data set and 100 notes as our test data set. We evaluated the performance of 10 deep learning models for detecting 80 symptoms included in the National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) framework. Model performance as compared with manual chart abstraction was assessed using standard metrics, and the highest performer was externally validated on a sample of 100 physician notes from a different clinical context.RESULTSIn our training and test data sets, 75 of the 80 candidate symptoms were identified. The ELECTRA-small model had the highest performance for symptom identification at the token level (ie, at the individual symptom level), with an F1 of 0.87 and a processing time of 3.95 seconds per note. For the 10 most common symptoms in the test data set, the F1 score ranged from 0.98 for anxious to 0.86 for fatigue. For external validation of the same symptoms, the note-level performance ranged from F1 = 0.97 for diarrhea and dizziness to F1 = 0.73 for swelling.CONCLUSIONTraining a deep learning model to identify a wide range of electronic health record–documented symptoms relevant to cancer care is feasible. This approach could be used at the health system scale to complement to electronic PROs.
JCO: Clinical Cancer Informatics – Wolters Kluwer Health
Published: Jun 17, 2022
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.