Automatic extraction of quantitative data from to conduct meta-analyses

J Clin Epidemiol. 2019 Jan;105:92-100. doi: 10.1016/j.jclinepi.2018.08.023. Epub 2018 Sep 23.


Objectives: Systematic reviews and meta-analyses are labor-intensive and time-consuming. Automated extraction of quantitative data from primary studies can accelerate this process., launched in 2000, is the world's largest trial repository of results data from clinical trials; it has been used as a source instead of journal articles. We have developed a Web application called EXACT (EXtracting Accurate efficacy and safety information from that allows users without advanced programming skills to automatically extract data from in analysis-ready format. We have also used the automatically extracted data to examine the reproducibility of meta-analyses in three published systematic reviews.

Study design and setting: We developed a Python-based software application (EXACT) that automatically extracts data required for meta-analysis from the database in a spreadsheet format. We confirmed the accuracy of the extracted data and then used those data to repeat meta-analyses in three published systematic reviews. To ensure that we used the same statistical methods and outcomes as the published systematic reviews, we repeated the meta-analyses using data manually extracted from the relevant journal articles. For the outcomes whose results we were able to reproduce using those journal article data, we examined the usability of data.

Results: EXACT extracted data at with 100% accuracy, and it required 60% less time than the usual practice of manually extracting data from journal articles. We found that 87% of the data elements extracted using EXACT matched those extracted manually from the journal articles. We were able to reproduce 24 of 28 outcomes using the journal article data. Of these 24 outcomes, we were able to reproduce 83.3% of the published estimates using data at

Conclusion: EXACT ( automatically and accurately extracted data elements from and thus reduced time in data extraction. The data reproduced most meta-analysis results in our study, but this conclusion needs further validation.

Keywords: Automatic data extraction;; Meta-analysis; Reproducibility; Simeprevir; Systematic review; Trametinib; Vortioxetine.

Publication types

  • Research Support, N.I.H., Extramural

MeSH terms

  • Clinical Trials as Topic*
  • Data Accuracy
  • Electronic Data Processing / methods*
  • Humans
  • Information Systems
  • Meta-Analysis as Topic*
  • Reproducibility of Results
  • Software*
  • Systematic Reviews as Topic*