Deciding not to measure performance: the case of acute otitis media

Jt Comm J Qual Saf. 2003 Jan;29(1):27-36. doi: 10.1016/s1549-3741(03)29004-3.


Background: No Health Plan Employer Data and Information Set (HEDIS) performance measures evaluate health plans for possible overuse or inappropriate use of antibiotics. Acute otitis media (AOM), or infection of the middle ear, is one of the most common infections in children. The antibiotic resistance of the bacteria that cause AOM and the general overuse and inappropriate use of antibiotics for this condition are taking center stage as a major public health threat. An effort was undertaken to develop a new HEDIS performance measure that evaluates appropriate antibiotic use in children with AOM.

The measure development process: The measure development process has three major phases: the initial development phase, the field-testing and analysis phase, and the measure refinement phase. With AOM, the measure development process could not proceed beyond the first phase for reasons that are be discussed in detail.

Conclusions: Additional difficulties beyond feasibility issues may arise in developing a quality of care performance measure. The measure development process discussed in this article failed primarily because of issues related to relevance as well as the scientific soundness of the proposed measures. If the evidence base related to the diagnosis and management of a particular condition is sparse, no standard can be developed against which to measure performance, and the criterion of scientific soundness cannot be met. This is the case with AOM. Unfortunately, the evidence base will likely remain inadequate to support the development of such a measure in the future.

Publication types

  • Evaluation Study
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Acute Disease
  • Anti-Bacterial Agents / pharmacology
  • Anti-Bacterial Agents / therapeutic use*
  • Child
  • Consensus
  • Drug Resistance*
  • Drug Utilization Review / standards*
  • Evidence-Based Medicine
  • Feasibility Studies
  • Health Benefit Plans, Employee / standards
  • Health Services Misuse
  • Humans
  • Otitis Media / diagnosis
  • Otitis Media / drug therapy*
  • Otitis Media / microbiology
  • Practice Guidelines as Topic
  • Quality Indicators, Health Care / classification
  • Quality Indicators, Health Care / standards*
  • United States


  • Anti-Bacterial Agents