Setting a fair performance standard for physicians' quality of patient care

J Gen Intern Med. 2011 May;26(5):467-73. doi: 10.1007/s11606-010-1572-x. Epub 2010 Nov 23.

Abstract

Background: Assessing physicians' clinical performance using statistically sound, evidence-based measures is challenging. Little research has focused on methodological approaches to setting performance standards to which physicians are being held accountable.

Objective: Determine if a rigorous approach for setting an objective, credible standard of minimally-acceptable performance could be used for practicing physicians caring for diabetic patients.

Design: Retrospective cohort study.

Participants: Nine hundred and fifty-seven physicians from the United States with time-limited certification in internal medicine or a subspecialty.

Main measures: The ABIM Diabetes Practice Improvement Module was used to collect data on ten clinical and two patient experience measures. A panel of eight internists/subspecialists representing essential perspectives of clinical practice applied an adaptation of the Angoff method to judge how physicians who provide minimally-acceptable care would perform on individual measures to establish performance thresholds. Panelists then rated each measure's relative importance and the Dunn-Rankin method was applied to establish scoring weights for the composite measure. Physician characteristics were used to support the standard-setting outcome.

Key results: Physicians abstracted 20,131 patient charts and 18,974 patient surveys were completed. The panel established reasonable performance thresholds and importance weights, yielding a standard of 48.51 (out of 100 possible points) on the composite measure with high classification accuracy (0.98). The 38 (4%) outlier physicians who did not meet the standard had lower ratings of overall clinical competence and professional behavior/attitude from former residency program directors (p = 0.01 and p = 0.006, respectively), lower Internal Medicine certification and maintenance of certification examination scores (p = 0.005 and p < 0.001, respectively), and primarily worked as solo practitioners (p = 0.02).

Conclusions: The standard-setting method yielded a credible, defensible performance standard for diabetes care based on informed judgment that resulted in a reasonable, reproducible outcome. Our method represents one approach to identifying outlier physicians for intervention to protect patients.

Publication types

  • Comparative Study
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adolescent
  • Adult
  • Aged
  • Clinical Competence / standards*
  • Cohort Studies
  • Diabetes Mellitus / diagnosis
  • Diabetes Mellitus / therapy
  • Employee Performance Appraisal / standards*
  • Female
  • Humans
  • Male
  • Middle Aged
  • Patient Care / methods
  • Patient Care / standards*
  • Physicians / standards*
  • Prospective Studies
  • Quality of Health Care / standards*
  • Retrospective Studies
  • Young Adult