Inference for optimal dynamic treatment regimes using an adaptive m-out-of-n bootstrap scheme

Biometrics. 2013 Sep;69(3):714-23. doi: 10.1111/biom.12052. Epub 2013 Jul 11.


A dynamic treatment regime consists of a set of decision rules that dictate how to individualize treatment to patients based on available treatment and covariate history. A common method for estimating an optimal dynamic treatment regime from data is Q-learning which involves nonsmooth operations of the data. This nonsmoothness causes standard asymptotic approaches for inference like the bootstrap or Taylor series arguments to breakdown if applied without correction. Here, we consider the m-out-of-n bootstrap for constructing confidence intervals for the parameters indexing the optimal dynamic regime. We propose an adaptive choice of m and show that it produces asymptotically correct confidence sets under fixed alternatives. Furthermore, the proposed method has the advantage of being conceptually and computationally much simple than competing methods possessing this same theoretical property. We provide an extensive simulation study to compare the proposed method with currently available inference procedures. The results suggest that the proposed method delivers nominal coverage while being less conservative than alternatives. The proposed methods are implemented in the qLearn R-package and have been made available on the Comprehensive R-Archive Network ( Analysis of the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) study is used as an illustrative example.

Keywords: Dynamic Treatment Regime; Nonregularity; Q-learning; m-Out-of-n Bootstrap.

Publication types

  • Research Support, N.I.H., Extramural

MeSH terms

  • Antidepressive Agents / therapeutic use
  • Artificial Intelligence
  • Biometry / methods*
  • Computer Simulation
  • Confidence Intervals
  • Decision Theory
  • Depressive Disorder, Major / drug therapy
  • Humans
  • Linear Models
  • Logistic Models
  • Models, Statistical
  • Monte Carlo Method
  • Randomized Controlled Trials as Topic / statistics & numerical data*


  • Antidepressive Agents