Variation in Variation Measurement: A Mapping Review of Methods to Study Clinical Variation

Abstract

Rationale: Variation in health care delivery exists at many levels (e.g., provider, practice, system) and can often be explained by various factors at each level. Understanding clinical variation presents an opportunity to improve the value of health care by identifying low-value care (overuse), gaps in high-value care (underuse), and how they can be improved. Numerous methods exist to describe or quantify clinical variation; however, these are not well identified or applied consistently.

Aim: A mapping review was used to identify and characterize available methods to describe and quantify clinical variation.

Method: We systematically searched health care and health services-related literature for variation and related terms used in titles and abstracts. Titles and abstracts were screened for inclusion. We then identified graphical and statistical methods used, health care specialty, study setting, and health system performance area (e.g., quality, access, costs) using a keyword analysis.

Results: Of the 16,969 papers screened, we excluded 10,866 that did not measure a care process or health outcome, measure variation at the person-level or higher, or analyze routinely collected data. We included 6,103 full-text studies, which were analyzed using a keyword analysis. Most studies used basic methodological approaches (e.g., regression, crude comparisons, ranges). Fewer than 1000 studies used multilevel models, a more advanced methodologic approach that quantifies the magnitude and source of variation. Multilevel models were not commonly used to study variation in health care quality.

Conclusions: While understanding clinical variation is important for all health systems, the methods used are usually able to identify but not quantify or explain variation. This review advances our knowledge of the scope and application of these methods and can be used to improve the measurement of variation to increase the value and equity of health care.

Keywords: clinical variation; funnel plots; health system performance; mapping review; medical practice variation; methods; multilevel models.