Extracting Family History Information From Electronic Health Records: Natural Language Processing Analysis

Maciej Rybinski; Xiang Dai; Sonit Singh; Sarvnaz Karimi; Anthony Nguyen

doi:10.2196/24020

Extracting Family History Information From Electronic Health Records: Natural Language Processing Analysis

JMIR Med Inform. 2021 Apr 30;9(4):e24020. doi: 10.2196/24020.

Authors

Maciej Rybinski¹, Xiang Dai^{1

2}, Sonit Singh^{1

3}, Sarvnaz Karimi¹, Anthony Nguyen⁴

Affiliations

¹ Commonwealth Scientific and Industrial Research Organisation, Sydney, Australia.
² University of Sydney, Sydney, Australia.
³ Macquarie University, Sydney, Australia.
⁴ Commonwealth Scientific and Industrial Research Organisation, Brisbane, Australia.

PMID: 33664015
PMCID: PMC8092929
DOI: 10.2196/24020

Abstract

Background: The prognosis, diagnosis, and treatment of many genetic disorders and familial diseases significantly improve if the family history (FH) of a patient is known. Such information is often written in the free text of clinical notes.

Objective: The aim of this study is to develop automated methods that enable access to FH data through natural language processing.

Methods: We performed information extraction by using transformers to extract disease mentions from notes. We also experimented with rule-based methods for extracting family member (FM) information from text and coreference resolution techniques. We evaluated different transfer learning strategies to improve the annotation of diseases. We provided a thorough error analysis of the contributing factors that affect such information extraction systems.

Results: Our experiments showed that the combination of domain-adaptive pretraining and intermediate-task pretraining achieved an F1 score of 81.63% for the extraction of diseases and FMs from notes when it was tested on a public shared task data set from the National Natural Language Processing Clinical Challenges (N2C2), providing a statistically significant improvement over the baseline (P<.001). In comparison, in the 2019 N2C2/Open Health Natural Language Processing Shared Task, the median F1 score of all 17 participating teams was 76.59%.

Conclusions: Our approach, which leverages a state-of-the-art named entity recognition model for disease mention detection coupled with a hybrid method for FM mention detection, achieved an effectiveness that was close to that of the top 3 systems participating in the 2019 N2C2 FH extraction challenge, with only the top system convincingly outperforming our approach in terms of precision.

Keywords: clinical natural language processing; data augmentation; information extraction; named entity recognition; natural language processing; neural language modeling; sequence tagging.

©Maciej Rybinski, Xiang Dai, Sonit Singh, Sarvnaz Karimi, Anthony Nguyen. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 30.04.2021.