Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
, 77, 11-20

Radiology Report Annotation Using Intelligent Word Embeddings: Applied to Multi-Institutional Chest CT Cohort

Affiliations

Radiology Report Annotation Using Intelligent Word Embeddings: Applied to Multi-Institutional Chest CT Cohort

Imon Banerjee et al. J Biomed Inform.

Abstract

We proposed an unsupervised hybrid method - Intelligent Word Embedding (IWE) that combines neural embedding method with a semantic dictionary mapping technique for creating a dense vector representation of unstructured radiology reports. We applied IWE to generate embedding of chest CT radiology reports from two healthcare organizations and utilized the vector representations to semi-automate report categorization based on clinically relevant categorization related to the diagnosis of pulmonary embolism (PE). We benchmark the performance against a state-of-the-art rule-based tool, PeFinder and out-of-the-box word2vec. On the Stanford test set, the IWE model achieved average F1 score 0.97, whereas the PeFinder scored 0.9 and the original word2vec scored 0.94. On UPMC dataset, the IWE model's average F1 score was 0.94, when the PeFinder scored 0.92 and word2vec scored 0.85. The IWE model had lowest generalization error with highest F1 scores. Of particular interest, the IWE model (trained on the Stanford dataset) outperformed PeFinder on the UPMC dataset which was used originally to tailor the PeFinder model.

Keywords: Information extraction; Pulmonary embolism; Report annotation; Word embedding.

Figures

Figure 1
Figure 1
Distribution of PE categorical measure: Stanford dataset (4512 reports) on left and UPMC dataset (858 reports) on right
Figure 2
Figure 2
Schema of Intelligent word embedding (IWE) approach
Figure 3
Figure 3
Ontocrawler pipeline
Figure 4
Figure 4
On left all word embeddings generated by IWE (vocabulary size - 3650 words) and visualized in two dimensions using t-SNE; On right clustering of the word embedding space using K-means++.
Figure 5
Figure 5
Unsupervised IWE reports embedding projected in 2D highlighting the label PE positive - Stanford test set (on left) and UPMC dataset (on right)
Figure 6
Figure 6
Unsupervised IWE reports embedding projected in 2D highlighting the label PE acute - Stanford test set (on left) and UPMC dataset (on right)
Figure 7
Figure 7
ROC Curve for IWE classifier - Stanford Test Set (on left) and UPMC Test set (on right)
Figure 8
Figure 8
Bar plots showing F1 scores in percentage computed on Stanford Test Set(on top) and UPMC Dataset(on bottom) where IWE is represented as dark brown, Out-of-box word2vec as sand, and PEFinder as white colored bar.

Similar articles

See all similar articles

Cited by 10 articles

See all "Cited by" articles

Publication types

LinkOut - more resources

Feedback