Estimating treatment effects with machine learning

Health Serv Res. 2019 Dec;54(6):1273-1282. doi: 10.1111/1475-6773.13212. Epub 2019 Oct 10.

Abstract

Objective: To demonstrate the performance of methodologies that include machine learning (ML) algorithms to estimate average treatment effects under the assumption of exogeneity (selection on observables).

Data sources: Simulated data and observational data on hospitalized adults.

Study design: We assessed the performance of several ML-based estimators, including Targeted Maximum Likelihood Estimation, Bayesian Additive Regression Trees, Causal Random Forests, Double Machine Learning, and Bayesian Causal Forests, applying these methods to simulated data as well as data on the effects of right heart catheterization.

Principal findings: In Monte Carlo studies, ML-based estimators generated estimates with smaller bias than traditional regression approaches, demonstrating substantial (69 percent-98 percent) bias reduction in some scenarios. Bayesian Causal Forests and Double Machine Learning were top performers, although all were sensitive to high dimensional (>150) sets of covariates.

Conclusions: ML-based methods are promising methods for estimating treatment effects, allowing for the inclusion of many covariates and automating the search for nonlinearities and interactions among variables. We provide guidance and sample code for researchers interested in implementing these tools in their own empirical work.

Keywords: machine learning; observational research; treatment effects.

Publication types

  • Observational Study
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adult
  • Aged
  • Aged, 80 and over
  • Algorithms
  • Bayes Theorem
  • Cardiac Catheterization / statistics & numerical data*
  • Computer Simulation*
  • Data Interpretation, Statistical*
  • Female
  • Humans
  • Inpatients / statistics & numerical data*
  • Machine Learning*
  • Male
  • Middle Aged
  • Models, Statistical
  • Treatment Outcome*