SIMPLifying Urology Residency Operative Assessments: A Pilot Study in Urology Training

J Urol. 2021 Oct;206(4):1009-1019. doi: 10.1097/JU.0000000000001874. Epub 2021 May 25.


Purpose: The Society for Improving Medical Professional Learning (SIMPL) app is an innovative, convenient and validated smartphone-based tool to evaluate residents' operative performance. In this study, we describe the initial implementation of SIMPL in our program's pediatric urology rotation-the first among urology residencies-and provide preliminary data on its adoption by residents and faculty.

Materials and methods: Residents and faculty in our pediatric urology division submitted SIMPL evaluations following surgical cases from August 2019 to July 2020. Evaluations consisted of ratings in 3 domains: resident autonomy, resident operative performance and patient-related case complexity. An online survey was also used to gauge attitudes towards SIMPL, describe patterns of use and solicit feedback on areas for improvement.

Results: Eight residents and 6 faculty submitted 141 evaluations, with 76.6% of evaluated cases having both faculty and resident ratings. Verbal feedback was included in 94.2%. Faculty-resident agreement ranged from 68.6% to 75.2% (kappa=0.47 to 0.61). Faculty rated postgraduate year (PGY)-4 residents as more autonomous (p=0.040) and higher performing (p=0.028) than PGY-3 residents. All participants agreed that SIMPL was easy to use and compared favorably to existing avenues of feedback. Barriers to implementation included lack of reminders for evaluations and evaluation fatigue.

Conclusions: The SIMPL application improved both frequency and quality of resident operative feedback. Among participants, SIMPL was preferred over the existing feedback system at our institution.

Keywords: education; feedback; internship and residency; pediatrics, educational assessment.

MeSH terms

  • Clinical Competence / statistics & numerical data
  • Faculty / statistics & numerical data
  • Feasibility Studies
  • Formative Feedback*
  • Humans
  • Internship and Residency / methods*
  • Internship and Residency / statistics & numerical data
  • Mobile Applications*
  • Pediatrics / education
  • Pilot Projects
  • Reproducibility of Results
  • Smartphone
  • Urologic Surgical Procedures / education*
  • Urologists / education
  • Urologists / statistics & numerical data
  • Urology / education*