Randomization, matching, and propensity scores in the design and analysis of experimental studies with measured baseline covariates

Abstract

In many experimental situations, researchers have information on a number of covariates prior to randomization. This information can be used to balance treatment assignment with respect to these covariates as well as in the analysis of the outcome data. In this paper, we investigate the use of propensity scores in both of these roles. We also introduce a randomization procedure in which the balance of all measured covariates is approximately indexed by the variance of the empirical propensity scores and randomization is restricted to those permutations with the least variable propensity scores. This procedure is compared with recently proposed methods in terms of resulting covariate balance and estimation efficiency. Properties of the estimators resulting from each procedure are compared with estimates which incorporate the propensity score in the analysis stage. Simulation results show that analytical adjustment for the propensity score yields results on par with those obtained through restricted randomization procedures and can be used in conjunction with such procedures to further improve inferential efficiency.

Keywords: Mahalanobis distance; covariate balance; experimental design; non-bipartite matching; propensity-constrained randomization; restricted randomization.