Assessment of Medicare's imaging efficiency measure for emergency department patients with atraumatic headache

Abstract

Study objective: Computed tomography (CT) use has increased rapidly, raising concerns about radiation exposure and cost. The Centers for Medicare & Medicaid Services (CMS) developed an imaging efficiency measure (Outpatient Measure 15 [OP-15]) to evaluate the use of brain CT in the emergency department (ED) for atraumatic headache. We aim to determine the reliability, validity, and accuracy of OP-15.

Methods: This was a retrospective record review at 21 US EDs. We identified 769 patient visits that CMS labeled as including an inappropriate brain CT to identify clinical indications for CT and reviewed the 748 visits with available records. The primary outcome was the reliability of OP-15 as determined by CMS from administrative data compared with medical record review. Secondary outcomes were the measure's validity and accuracy. Outcome measures were defined according to the testing protocol of the American Medical Association's Physician Consortium for Performance Improvement.

Results: On record review, 489 of 748 ED brain CTs identified as inappropriate by CMS had a measure exclusion documented that was not identified by administrative data; the measure was 34.6% reliable (95% confidence interval [CI] 31.2% to 38.0%). Among the 259 patient visits without measure exclusions documented in the record, the measure's validity was 47.5% (95% CI 41.4% to 53.6%), according to a consensus list of indications for brain CT. Overall, 623 of the 748 ED visits had either a measure exclusion or a consensus indication for CT; the measure's accuracy was 16.7% (95% CI 14% to 19.4%). Hospital performance as reported by CMS did not correlate with the proportion of CTs with a documented clinical indication (r=-0.11; P=.63).

Conclusion: The CMS imaging efficiency measure for brain CTs (OP-15) is not reliable, valid, or accurate and may produce misleading information about hospital ED performance.