Multimethod evaluation of health policy change: an application to Medicaid managed care in a rural state

Abstract

Objective: To answer questions about the impacts of Medicaid managed care (MMC) at the individual, organizational/community, and population levels of analysis.

Data sources/study setting: Multimethod approach to study MMC in New Mexico, a rural state with challenging access barriers.

Study design: Individual level: surveys to assess barriers to care, access, utilization, and satisfaction. Organizational/community level: ethnography to determine changes experienced by safety net institutions and local communities. Population level: analysis of secondary databases to examine trends in preventable adverse sentinel events.

Survey: multivariate statistical methods, including factor analysis and logistic regression. Ethnography: iterative coding and triangulation to assess documents, field observations, and in-depth interviews. Secondary databases: plots of sentinel events over time.

Principal findings: The survey component revealed no consistent changes after MMC, relatively favorable experiences for Medicaid patients, and persisting access barriers for the uninsured. In the ethnographic component, safety net institutions experienced increased workload and financial stress; mental health services declined sharply. Immunization rate, as an important sentinel event, deteriorated.

Conclusions: MMC exerted greater effects on safety net providers than on individuals and did not address problems of the uninsured. A multimethod approach can facilitate evaluation of change in health policy.

Figure 1

Conceptual and Methodological Model of the Effects of Medicaid Managed Care Reform.

Figure 2

Changes in Key Sentinel Events. (A) Prenatal Care. (B) Immunization Coverage Level Estimates. Note: (a) Source: New Mexico birth certificates. Results for high-income and low-income zip codes are shown. Arrow marks the date of implementation for Medicaid managed care. Note: (b) Source: National Immunization Survey, 1996–2000. Data for the 4:3:1 immunization series (four doses of diphtheria, tetanus, and acellular pertussis vaccines; three doses of poliovirus vaccine; and one dose of measles vaccine) are shown. Arrow marks the date of implementation for Medicaid managed care.