Q- and A-learning Methods for Estimating Optimal Dynamic Treatment Regimes

Stat Sci. 2014 Nov;29(4):640-661. doi: 10.1214/13-STS450.


In clinical practice, physicians make a series of treatment decisions over the course of a patient's disease based on his/her baseline and evolving characteristics. A dynamic treatment regime is a set of sequential decision rules that operationalizes this process. Each rule corresponds to a decision point and dictates the next treatment action based on the accrued information. Using existing data, a key goal is estimating the optimal regime, that, if followed by the patient population, would yield the most favorable outcome on average. Q- and A-learning are two main approaches for this purpose. We provide a detailed account of these methods, study their performance, and illustrate them using data from a depression study.

Keywords: Advantage learning; bias-variance tradeoff; model misspecification; personalized medicine; potential outcomes; sequential decision making.