Implementation and Initial Construct Validity Evidence of a Tool, myTIPreport, for Interactive Workplace Feedback on ACGME Milestones

J Surg Educ. 2020 Jun 13;S1931-7204(20)30137-9. doi: 10.1016/j.jsurg.2020.05.002. Online ahead of print.

Abstract

Objective: To describe implementation of myTIPreport for milestone feedback and to initiate construct validity testing of myTIPreport for milestones.

Design: myTIPreport was used to provide workplace feedback on Accreditation Council for Graduate Medical Education required milestone sets. Performance of senior learners (postgraduate year [PGY]-4s) was compared to that of junior learners (PGY-1s) to begin the process of construct validity testing for myTIPreport.

Setting: A convenience-based site selection of Obstetrics and Gynecology (OBGYN) residency programs.

Participants: OBGYN residents and faculty.

Results: Amongst the 12 participating OBGYN residency programs, there were 444 unique learners and 343 unique faculty teachers. A total of 5293 milestone feedback encounters were recorded. Mean PGY-4 performance was rated higher than mean PGY-1 performance on all 25 of the compared milestone sets, with statistically significant differences seen for 19 (76%) of these 25 milestone sets and nonsignificant differences in the predicted direction observed for the other 6 milestone sets.

Conclusions: myTIPreport detected differences between senior and junior learners for the majority of compared feedback encounters for OBGYN residents. Findings support the emerging construct validity of myTIPreport for milestone feedback.

Keywords: Medical education; Milestones; Real-time feedback; Resident education.