Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2017 May 15;19(5):e162.
doi: 10.2196/jmir.6887.

Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study

Stephane Meystre  1 , Ramkiran Gouripeddi  2 , Joel Tieder  3 , Jeffrey Simmons  4 , Rajendu Srivastava  5   6 , Samir Shah  4
Affiliations
Multicenter Study

Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study

Stephane Meystre et al. J Med Internet Res. .

Abstract

Background: Community-acquired pneumonia is a leading cause of pediatric morbidity. Administrative data are often used to conduct comparative effectiveness research (CER) with sufficient sample sizes to enhance detection of important outcomes. However, such studies are prone to misclassification errors because of the variable accuracy of discharge diagnosis codes.

Objective: The aim of this study was to develop an automated, scalable, and accurate method to determine the presence or absence of pneumonia in children using chest imaging reports.

Methods: The multi-institutional PHIS+ clinical repository was developed to support pediatric CER by expanding an administrative database of children's hospitals with detailed clinical data. To develop a scalable approach to find patients with bacterial pneumonia more accurately, we developed a Natural Language Processing (NLP) application to extract relevant information from chest diagnostic imaging reports. Domain experts established a reference standard by manually annotating 282 reports to train and then test the NLP application. Findings of pleural effusion, pulmonary infiltrate, and pneumonia were automatically extracted from the reports and then used to automatically classify whether a report was consistent with bacterial pneumonia.

Results: Compared with the annotated diagnostic imaging reports reference standard, the most accurate implementation of machine learning algorithms in our NLP application allowed extracting relevant findings with a sensitivity of .939 and a positive predictive value of .925. It allowed classifying reports with a sensitivity of .71, a positive predictive value of .86, and a specificity of .962. When compared with each of the domain experts manually annotating these reports, the NLP application allowed for significantly higher sensitivity (.71 vs .527) and similar positive predictive value and specificity .

Conclusions: NLP-based pneumonia information extraction of pediatric diagnostic imaging reports performed better than domain experts in this pilot study. NLP is an efficient method to extract information from a large collection of imaging reports to facilitate CER.

Keywords: comparative effectiveness research; medical informatics; natural language processing; pneumonia, bacterial.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Diagnostic imaging report annotations example.
Figure 2
Figure 2
Examples of domain expert annotation disagreements.
Figure 3
Figure 3
Components of the pneumonia clinical information extraction application.
Figure 4
Figure 4
Pruned decision tree for pneumonia classification.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Lee GE, Lorch SA, Sheffler-Collins S, Kronman MP, Shah SS. National hospitalization trends for pediatric pneumonia and associated complications. Pediatrics. 2010 Aug;126(2):204–13. doi: 10.1542/peds.2009-3109. http://europepmc.org/abstract/MED/20643717 - DOI - PMC - PubMed
    1. Keren R, Luan X, Localio R, Hall M, McLeod L, Dai D, Srivastava R, Pediatric Research in Inpatient Settings (PRIS) Network Prioritization of comparative effectiveness research topics in hospital pediatrics. Arch Pediatr Adolesc Med. 2012 Dec;166(12):1155–64. doi: 10.1001/archpediatrics.2012.1266. - DOI - PubMed
    1. Bradley JS, Byington CL, Shah SS, Alverson B, Carter ER, Harrison C, Kaplan SL, Mace SE, McCracken GH, Moore MR, St Peter SD, Stockwell JA, Swanson JT, Pediatric Infectious Diseases Societythe Infectious Diseases Society of America The management of community-acquired pneumonia in infants and children older than 3 months of age: clinical practice guidelines by the Pediatric Infectious Diseases Society and the Infectious Diseases Society of America. Clin Infect Dis. 2011 Oct;53(7):e25–76. doi: 10.1093/cid/cir531. - DOI - PMC - PubMed
    1. Ambroggio L, Taylor JA, Tabb LP, Newschaffer CJ, Evans AA, Shah SS. Comparative effectiveness of empiric β-lactam monotherapy and β-lactam-macrolide combination therapy in children hospitalized with community-acquired pneumonia. J Pediatr. 2012 Dec;161(6):1097–103. doi: 10.1016/j.jpeds.2012.06.067. - DOI - PubMed
    1. Williams DJ, Hall M, Shah SS, Parikh K, Tyler A, Neuman MI, Hersh AL, Brogan TV, Blaschke AJ, Grijalva CG. Narrow vs broad-spectrum antimicrobial therapy for children hospitalized with pneumonia. Pediatrics. 2013 Nov;132(5):e1141–8. doi: 10.1542/peds.2013-1614. http://pediatrics.aappublications.org/cgi/pmidlookup?view=long&pmid=2416... - DOI - PMC - PubMed

Publication types