Machine learning model combining features from algorithms with different analytical methodologies to detect laboratory-event-related adverse drug reaction signals

PLoS One. 2018 Nov 21;13(11):e0207749. doi: 10.1371/journal.pone.0207749. eCollection 2018.


Background: The importance of identifying and evaluating adverse drug reactions (ADRs) has been widely recognized. Many studies have developed algorithms for ADR signal detection using electronic health record (EHR) data. In this study, we propose a machine learning (ML) model that enables accurate ADR signal detection by integrating features from existing algorithms based on inpatient EHR laboratory results.

Materials and methods: To construct an ADR reference dataset, we extracted known drug-laboratory event pairs represented by a laboratory test from the EU-SPC and SIDER databases. All possible drug-laboratory event pairs, except known ones, are considered unknown. To detect a known drug-laboratory event pair, three existing algorithms-CERT, CLEAR, and PACE-were applied to 21-year inpatient EHR data. We also constructed ML models (based on random forest, L1 regularized logistic regression, support vector machine, and a neural network) that use the intermediate products of the CERT, CLEAR, and PACE algorithms as inputs and determine whether a drug-laboratory event pair is associated. For performance comparison, we evaluated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F1-measure, and area under receiver operating characteristic (AUROC).

Results: All measures of ML models outperformed those of existing algorithms with sensitivity of 0.593-0.793, specificity of 0.619-0.796, NPV of 0.645-0.727, PPV of 0.680-0.777, F1-measure of 0.629-0.709, and AUROC of 0.737-0.816. Features related to change or distribution of shape were considered important for detecting ADR signals.

Conclusions: Improved performance of ML models indicated that applying our model to EHR data is feasible and promising for detecting more accurate and comprehensive ADR signals.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Drug-Related Side Effects and Adverse Reactions / diagnosis*
  • Laboratories*
  • Machine Learning*
  • Software

Grant support

This research was supported by grants from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI) funded by the Ministry of Health & Welfare, Republic of Korea (grant numbers: HI16C0982, HI17C0970, and HG18C0067). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.