On the joint use of propensity and prognostic scores in estimation of the average treatment effect on the treated: a simulation study

Abstract

Propensity and prognostic score methods seek to improve the quality of causal inference in non-randomized or observational studies by replicating the conditions found in a controlled experiment, at least with respect to observed characteristics. Propensity scores model receipt of the treatment of interest; prognostic scores model the potential outcome under a single treatment condition. While the popularity of propensity score methods continues to grow, prognostic score methods and methods combining propensity and prognostic scores have thus far received little attention. To this end, we performed a simulation study that compared subclassification and full matching on a single estimated propensity or prognostic score with three approaches combining the estimated propensity and prognostic scores: full matching on a Mahalanobis distance combining the estimated propensity and prognostic scores (FULL-MAHAL); full matching on the estimated prognostic propensity score within propensity score calipers (FULL-PGPPTY); and subclassification on an estimated propensity and prognostic score grid with 5 × 5 subclasses (SUBCLASS(5*5)). We considered settings in which one, both, or neither score model was misspecified. The data generating mechanisms varied in the degree of linearity and additivity in the true treatment assignment and outcome models. FULL-MAHAL and FULL-PGPPTY exhibited strong to superior performance in root mean square error terms across all simulation settings and scenarios. Methods combining propensity and prognostic scores were no less robust to model misspecification than single-score methods even when both score models were incorrectly specified. Our findings support the joint use of propensity and prognostic scores in estimation of the average treatment effect on the treated.

Keywords: Mahalanobis metric; bias reduction; confounding; full matching; observational data; subclassification.

Figure 1

Simulation Study Data Structure. (Reproduced with minor alterations from Lee et al. [44])

Figure 2

Percent Bias in ATT estimate by propensity/prognostic score estimation method for an incorrectly specified propensity score model, N=2000

Figure 3

95% Confidence Interval Coverage by propensity/prognostic score estimation method for an incorrectly specified propensity score model, N=2000

Figure 4

Root Mean Square Error of ATT estimate by propensity/prognostic score estimation method for an incorrectly specified propensity score model, N=2000

Figure 5

Percent Bias in ATT estimate by propensity/prognostic score estimation method for an incorrectly specified prognostic score model, N=2000

Figure 6

95% Confidence Interval Coverage by propensity/prognostic score estimation method for an incorrectly specified prognostic score model, N=2000

Figure 7

Root Mean Square Error of ATT estimate by propensity/prognostic score estimation method for an incorrectly specified prognostic score model, N=2000

Figure 8

Root Mean Square Error of ATT estimate by propensity/prognostic score estimation method for incorrectly specified propensity and prognostic score models, N=2000

Figure 9

Root Mean Square Error of ATT estimate by propensity/prognostic score estimation method for correctly specified propensity and prognostic score models, N=2000