Bias in telephone surveys that do not sample cell phones: uses and limits of poststratification adjustments

Abstract

Objective: To examine how biased health surveys are when they omit cell phone-only households (CPOH) and to explore whether poststratification can reduce this bias.

Methods: We used data from the 2008 National Health Interview Survey (NHIS), which uses area probability sampling and in-person interviews; as a result people of all phone statuses are included. First, we examined whether people living in CPOH are different from those not living in CPOH with respect to several important health surveillance domains. We compared standard NHIS estimates to a set of "reweighted" estimates that exclude people living in CPHO. The reweighted NHIS cases were fitted through a series of poststratification adjustments to NHIS control totals. In addition to poststratification adjustments for region, race or ethnicity, and age, we examined adjustments for home ownership, age by education, and household structure.

Results: Poststratification reduces bias in all health-related estimates for the nonelderly population. However, these adjustments work less well for Hispanics and blacks and even worse for young adults (18 to 30 y). Reduction in bias is greatest for estimates of uninsurance and having no usual source of care, and worse for estimates of drinking, smoking, and forgone or delayed care because of costs.

Conclusions: Applying poststratification adjustments to data that exclude CPOH works well at the total population level for estimates such as health insurance, and less well for access and health behaviors. However, poststratification adjustments do not do enough to reduce bias in health-related estimates at the subpopulation level, particularly for those interested in measuring and monitoring racial, ethnic, and age disparities.