Association Between Entrustable Professional Activities and Milestones Evaluations: Real-time Assessments Correlate With Semiannual Reviews

Abstract

Objective: Entrustable professional activities (EPAs) have been developed to refine competency-based education. The American Board of Surgery has initiated a 2-year pilot study to evaluate the impact of EPAs on the evaluation and feedback of surgical residents. The ACGME Milestones in Surgery is a semiannual competency-based evaluation program to measure resident progression through 16 professional attributes across 8 practice domains. The correlation between these 2 evaluation tools remains unclear. The purpose of this study is to evaluate this correlation through comparison of an EPA with the corresponding elements of the ACGME Milestones.

Design: From July, 2018 to October, 2019, all residents submitting EPA evaluations for gall bladder disease were evaluated for preoperative, intraoperative, and/or postoperative entrustability. The ratings were converted to a numerical rank from 0 to 4. Milestones scores from May 2019 and November 2019 were obtained for each resident, with scores ranging from 0 to 4. The gall bladder EPA incorporates the operative PC3 and MK2 and nonoperative PC1, PC2, and ICS3 components. Spearman rank correlation was conducted to evaluate the association between each resident's median EPA ranking and his/her milestones scores.

Setting: SUNY Upstate Medical University, Syracuse, NY, a university-based hospital.

Participants: General surgery residents.

Results: Among 24 residents, 106 intraoperative EPA evaluations were. For both the May and November milestones, significant positive correlations were noted for PC3 (correlation coefficient ρ = 0.690, p < 0.001; ρ = 0.876, p < 0.001). Similarly, for MK2, a significant positive correlation was noted (ρ = 0.882, p < 0.001; ρ = 0.759, p < 0.001). Interestingly, significant positive correlations were also identified between the 3 nonoperative milestones and the intraoperative entrustability ranking.

Conclusions: We observed significant correlations between EPAs for cholecystectomy and associated milestones evaluation scores. These findings indicate that EPAs may provide more timely and specific feedback than existing tools and, on aggregate, may improve upon existing formative feedback practices provided through the biannual evaluation of surgical residents.

Keywords: Assessment; Competency-based education; Entrustable professional activities; Entrustment; Evaluation; Milestones.