Impact of statistical approaches for handling missing data on trauma center quality

Abstract

Objective: To determine whether imputed data can be used to produce unbiased hospital quality measures.

Background: Different methods for handling missing data may influence which hospitals are designated as quality outliers.

Methods: Monte-Carlo simulation study based on 63,020 patients with no missing data in 68 hospitals using the National Trauma Data Bank (NTDB, version 6.1). Patients were assigned missing data for the motor component of the Glasgow coma scale (GCS) conditional on their observed clinical risk factors. Multiple imputation was then used to "fill in" the missing data. Hospital risk-adjusted quality measures (observed-to-expected mortality ratio) based either on (1) imputed data, (2) excluding patients with missing data (complete case analysis), or (3) excluding the predictor with missing data were compared with hospital quality based on the true data (no missing data). Pair-wise comparisons of hospital quality were performed using the intraclass correlation coefficient (ICC) and the kappa statistic.

Results: With 10% of the data missing, the level of agreement between multiple imputation and the true data (ICC = 0.99 and kappa = 0.87) was better compared with the level of agreement between complete case analysis and the true data (ICC = 0.93 and kappa = 0.62). Excluding the predictor (motor GCS) with missing data from the risk adjustment model resulted in the least amount of agreement with quality assessment based on the true data (ICC = 0.88 and kappa = 0.46).

Conclusion: Multiple imputation can be used to impute missing data and yields hospital quality measures that are nearly identical to those based on the true data. Simply excluding patients with missing data or excluding risk factors with missing data from hospital quality assessment yields substantially inferior quality measures.