Validly interpreting patients' reports: using bifactor and multidimensional models to determine whether surveys and scales measure one or more constructs

Abstract

Introduction: Validly interpreting scores based on patients' reports of their experiences using scales depends on a valid scoring system. The possibility exists that, although the creators of a scale intended to create a scale measuring a single construct, the questions may seem to measure more than one construct. However, the possibility also exists that the questions do generally measure one construct, but several "nuisance" constructs influence the measurement of the construct indirectly. Unidimensional, multidimensional, and bifactor analytical models offer a method for establishing scale dimensionality and subsequently developing valid scoring systems. In this paper, we take an applied perspective, discuss these models and their implications, and use real data to provide an example.

Methods: We used unidimensional, multidimensional, and bifactor analyses to examine the measurement structure of the Consumer Assessments of Healthcare Providers and Systems Cultural Competence (CAHPS-CC) Survey. Participants came from a 2008 sample of 2 Medicaid managed care plans, in New York and California.

Results: Both unidimensional and bifactor models failed to fit the data well. A multidimensional model, with 7 factors corresponding to 7 cultural competence domains fit the data well (root mean square error of approximation=0.064; Tucker-Lewis Index=0.98; comparative fit index=0.97).

Discussion: Results indicate that the CAHPS-CC does not seem to measure a single construct. Rather, the CAHPS-CC seems to measure 7 separable domains, suggesting the need for 7 separate scale scores. Our findings highlight the importance of conducting dimensionality analyses.