Assessing achievement in nephrology training: using clinic chart audits to quantitatively screen competency

Am J Kidney Dis. 2014 Nov;64(5):737-43. doi: 10.1053/j.ajkd.2014.06.027. Epub 2014 Aug 23.

Abstract

Background: Entrustable professional activities (EPAs) are complex tasks representing vital physician functions in multiple competencies, used to demonstrate trainee development along milestones. Managing a nephrology outpatient clinic has been proposed as an EPA for nephrology fellowship training.

Study design: Retrospective cohort study of nephrology fellow outpatient clinic performance using a previously validated chart audit tool.

Setting & participants: Outpatient encounter chart audits for training years 2008-2009 through 2012-2013, corresponding to participation in the Nephrology In-Training Examination (ITE). A median of 7 auditors (attending nephrologists) audited a mean of 1,686±408 (SD) charts per year. 18 fellows were audited; 12, in both of their training years.

Predictors: Proportion of chart audit and quality indicator deficiencies.

Outcomes: Longitudinal deficiency and ITE performance.

Measurements & results: Among fellows audited in both their training years, chart audit deficiencies were fewer in the second versus the first year (5.4%±2.0% vs 17.3%±7.0%; P<0.001) and declined between the first and second halves of the first year (22.2%±6.4% vs 12.3%±9.5%; P=0.002). Most deficiencies were omission errors, regardless of training year. Quality indicator deficiencies for hypertension and chronic kidney disease-associated anemia recognition and management were fewer during the second year (P<0.001). Yearly audit deficiencies ≥5% were associated with an ITE score less than the 25th percentile for second-year fellows (P=0.03), with no significant association for first-year fellows. Auditor-reported deficiencies declined between the first and second halves of the year (17.0% vs 11.1%; P<0.001), with a stable positive/neutral comment rate (17.3% vs 17.8%; P=0.6), suggesting that the decline was not due to auditor fatigue.

Limitations: Retrospective design and small trainee numbers.

Conclusions: Managing a nephrology outpatient clinic is an EPA. The chart audit tool was used to assess longitudinal fellow performance in managing a nephrology outpatient clinic. Failure to progress may be quantitatively identified and remediated. The tool identifies deficiencies in all 6 competencies, not just medical knowledge, the primary focus of the ITE and the nephrology subspecialty board examination.

Keywords: Chart audit; competencies; educational outcomes; entrustable professional activities (EPAs); fellowship; graduate medical education; milestones; nephrology; training programs.

Publication types

  • Evaluation Study

MeSH terms

  • Achievement*
  • Clinical Audit / methods
  • Clinical Audit / standards*
  • Clinical Competence / standards*
  • Cohort Studies
  • Humans
  • Internship and Residency / standards*
  • Nephrology / education*
  • Nephrology / standards*
  • Outpatient Clinics, Hospital / standards
  • Retrospective Studies