Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Feb;49(1):284-303.
doi: 10.1111/1475-6773.12090. Epub 2013 Jul 16.

Generalizing observational study results: applying propensity score methods to complex surveys

Eva H Dugoff  1 , Megan Schuler, Elizabeth A Stuart
Affiliations

Generalizing observational study results: applying propensity score methods to complex surveys

Eva H Dugoff et al. Health Serv Res. 2014 Feb.

Abstract

Objective: To provide a tutorial for using propensity score methods with complex survey data.

Data sources: Simulated data and the 2008 Medical Expenditure Panel Survey.

Study design: Using simulation, we compared the following methods for estimating the treatment effect: a naïve estimate (ignoring both survey weights and propensity scores), survey weighting, propensity score methods (nearest neighbor matching, weighting, and subclassification), and propensity score methods in combination with survey weighting. Methods are compared in terms of bias and 95 percent confidence interval coverage. In Example 2, we used these methods to estimate the effect on health care spending of having a generalist versus a specialist as a usual source of care.

Principal findings: In general, combining a propensity score method and survey weighting is necessary to achieve unbiased treatment effect estimates that are generalizable to the original survey target population.

Conclusions: Propensity score methods are an essential tool for addressing confounding in observational studies. Ignoring survey weights may lead to results that are not generalizable to the survey target population. This paper clarifies the appropriate inferences for different propensity score methods and suggests guidelines for selecting an appropriate propensity score method based on a researcher's goal.

Keywords: Survey research; health care costs; primary care.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Recommended Decision Tree for Researchers Interested in Causal Estimand Using Complex Survey Data
Figure 2
Figure 2
Estimated Absolute Bias and 95 percent Confidence Interval Coverage Rates When Estimating the PATE and PATT. (Methods that combine propensity score methods and survey weights are depicted with black bars. Naïve, no survey weight or propensity score; SW, survey weight; Weight, Propensity weight; Sub, Propensity score subclassification; NN, 1 : 1 matching; WeightSW, Survey weight and propensity weight; SubSW, subclassification and survey weight; NNSW, 1 : 1 matching and survey weight)
Figure 3
Figure 3
A Comparison of the Absolute Standardized Differences in Means for Covariates When Estimating the PATE, PATT, SATE, and SATT Using Different Propensity Score Methods
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Austin PC, Mamdani MM. A Comparison of Propensity Score Methods: A Case-Study Estimating the Effectiveness of Post-AMI Statin Use. Statistics in Medicine. 2006;25:2084–106. - PubMed
    1. Bell BA, Onwuegbuzie AJ, Ferron JM, Jiao QG, Hibbard ST, Kromrey JD. Use of Design Effects and Sample Weights in Complex Health Survey Data: A Review of Published Articles Using Data from Three Commonly Used Adolescent Health Surveys. American Journal of Public Health. 2012;102(7):1399–405. - PMC - PubMed
    1. Cain LE, Cole SR. Inverse Probability-of-Censoring Weights for the Correction of Time-Varying Noncompliance in the Effect of Randomized Highly Active Antiretroviral Therapy on Incident AIDS or Death. Statistics in Medicine. 2009;28:1725–38. - PubMed
    1. Cochran WG. The Effectiveness of Adjustment by Subclassification in Removing Bias in Observational Studies. Biometrics. 1968;24(2):295–313. - PubMed
    1. Cole SR, Hernan MA, Robins JM, Asastos K, Chmiel J, Detels R, Ervin C, Feldman J, Greenblatt R, Kingsley L, Lai S, Young M, Cohen M, Muñoz A. Effect of Highly Active Antiretroviral Therapy on Time to Acquired Immunodeficiency Syndrome or Death Using Marginal Structural Models. American Journal of Epidemiology. 2003;158(7):687–94. - PubMed

Publication types

LinkOut - more resources