Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling study

Abstract

Risk-adjustment schemes are used to monitor hospital performance, on the assumption that excess mortality not explained by case mix is largely attributable to suboptimal care. We have developed a model to estimate the proportion of the variation in standardised mortality ratios (SMRs) that can be accounted for by variation in preventable mortality. The model was populated with values from the literature to estimate a predictive value of the SMR in this context-specifically the proportion of those hospitals with SMRs among the highest 2.5% that fall among the worst 2.5% for preventable mortality. The extent to which SMRs reflect preventable mortality rates is highly sensitive to the proportion of deaths that are preventable. If 6% of hospital deaths are preventable (as suggested by the literature), the predictive value of the SMR can be no greater than 9%. This value could rise to 30%, if 15% of deaths are preventable. The model offers a 'reality check' for case mix adjustment schemes designed to isolate the preventable component of any outcome rate.

Figure 1

Diagnostic performance of the standardised mortality ratio (2-sigma upper limit) to detect a hospital among the worst 2.5% for preventable deaths. The curve shows the dependency of the upper bound for positive predictive value (PPV) (or true positive rate (TPR)) on the preventability index under a risk-adjustment scheme accounting for 80% of the variation between hospitals. The base case relationship c_V = 2c_M is assumed.

Figure 2

Three candidate distributions to describe variation among hospitals in rates of preventable mortality. The distributions are scaled to unit median and a log-normal model is assumed. Under the base case (c_V=0.4) the hospital in the 95th centile would have about four times the preventable mortality rate of the hospital at the 5th centile. Under the most dispersed distribution (c_V=1.0) the ratio between the 5th and 95th centiles (ie, 0.25 and 3.93) is more than 15, an implausibly large range across a random sample of 20 hospitals.