Impact of Cross-level Measurement Noninvariance on Hospital Rankings Based on Patient Experiences With Care in 7 European Countries

Abstract

Background: Hospital-level findings on patient experiences with care are increasingly reported publicly. A critical aspect left unexamined is the commonality of composite measures of patient experiences across different groups of patients, nursing units, hospitals, and countries. Absence of commonality is termed measurement noninvariance and is hypothesized to have a strong impact on performance assessment.

Aim: The aim of this study is to examine measurement invariance across groups and levels under study (patients, nursing units, hospitals, and countries) and illustrate the degree to which this method of analysis impacts hospital rankings.

Research design: Data were collected from 11,289 patients in 7 European countries, 186 hospitals, and 824 nursing units. Multilevel factor analytic models were applied to evaluate measurement invariance across the hierarchical levels of the study and across groups at specific levels (self-perceived health at patient level; unit speciality at nursing unit level). Hospital rankings for the final multilevel model were compared with those from a single-level factor model that is unsuspecting of measurement invariance.

Results: Cross-group invariance was shown for levels of self-perceived health and to a large degree also for nursing unit speciality. Patient experience composite measures were, however, not invariant across patient, unit, and hospital levels. Hospital rankings were largely impacted when accounted for this cross-level invariance. The percentage of hospitals with discordant ranks by >10 percentile points varied from 26.7% in Spain to 70% in Poland.

Conclusions: Leaving unexamined possible noninvariance across groups and hierarchical levels may have far reaching consequences for how the public perceives hospitals' position relative to other hospitals.