Variation in Identifying Sepsis and Organ Dysfunction Using Administrative Versus Electronic Clinical Data and Impact on Hospital Outcome Comparisons

Abstract

Objectives: Administrative claims data are commonly used for sepsis surveillance, research, and quality improvement. However, variations in diagnosis, documentation, and coding practices for sepsis and organ dysfunction may confound efforts to estimate sepsis rates, compare outcomes, and perform risk adjustment. We evaluated hospital variation in the sensitivity of claims data relative to clinical data from electronic health records and its impact on outcome comparisons.

Design, setting, and patients: Retrospective cohort study of 4.3 million adult encounters at 193 U.S. hospitals in 2013-2014.

Interventions: None.

Measurements and main results: Sepsis was defined using electronic health record-derived clinical indicators of presumed infection (blood culture draws and antibiotic administrations) and concurrent organ dysfunction (vasopressors, mechanical ventilation, doubling in creatinine, doubling in bilirubin to ≥ 2.0 mg/dL, decrease in platelets to < 100 cells/µL, or lactate ≥ 2.0 mmol/L). We compared claims for sepsis prevalence and mortality rates between both methods. All estimates were reliability adjusted to account for random variation using hierarchical logistic regression modeling. The sensitivity of hospitals' claims data was low and variable: median 30% (range, 5-54%) for sepsis, 66% (range, 26-84%) for acute kidney injury, 39% (range, 16-60%) for thrombocytopenia, 36% (range, 29-44%) for hepatic injury, and 66% (range, 29-84%) for shock. Correlation between claims and clinical data was moderate for sepsis prevalence (Pearson coefficient, 0.64) and mortality (0.61). Among hospitals in the lowest sepsis mortality quartile by claims, 46% shifted to higher mortality quartiles using clinical data. Using implicit sepsis criteria based on infection and organ dysfunction codes also yielded major differences versus clinical data.

Conclusions: Variation in the accuracy of claims data for identifying sepsis and organ dysfunction limits their use for comparing hospitals' sepsis rates and outcomes. Using objective clinical data may facilitate more meaningful hospital comparisons.

Figure 1.. Sensitivity of A) sepsis codes and B) organ dysfunction codes relative to clinical criteria

Boxes indicate the median hospital sensitivity (middle line), 25^th quartile (lower box line), and 75^th quartile (upper box line). Diamonds indicate the mean hospital sensitivity. Outliers are indicated by X (near outliers) and O (far outliers), as defined by values more than 1.5 times the interquartile range from the interior quartile boxes. The organ dysfunction comparisons in Figure 1B were conducted in hospitalizations with ≥1 blood culture draw.

Figure 2.. Histogram of hospital sepsis mortality rates by explicit sepsis codes and clinical criteria

All mortality rates are reliability-adjusted.

Figure 3.. Hospital sepsis mortality rates ranked by clinical criteria and compared to claims data

Hospitals are ranked from left to right according to mortality rates for sepsis as defined by clinical criteria (triangles). For each hospital, the corresponding sepsis mortality by explicit sepsis codes (circles) and implicit codes (squares) is displayed. All mortality rates are reliability-adjusted.

Figure 4.. Concordance of hospital sepsis mortality rates when ranked into quartiles by A) explicit sepsis codes and B) implicit sepsis codes versus clinical criteria.

Bubble sizes are proportional to the number of hospitals in each matched quartile. The actual number of hospitals in each category is denoted within the bubbles. The dotted line indicates where all hospitals / bubbles would lie if concordance was perfect between claims and clinical definitions. Lower quartiles indicate better performance (i.e., quartile 1 = lowest sepsis mortality, quartile 4 = highest sepsis mortality). Black bubbles = mortality quartiles that match between clinical criteria versus claims data. Grey bubbles above the dotted line = quartiles that are worse by clinical criteria versus claims data. Grey bubbles below the dotted line = quartiles that are better by clinical criteria versus claims data.