Objective: The Accreditation Council for Graduate Medical Education (ACGME) has mandated that residency programs document progression of competency-based outcomes. The Ophthalmic Clinical Evaluation Exercise (OCEX) assesses clinical skills in ophthalmology residents during patient encounters. Although OCEX has been validated for assessing several of the ACGME-mandated competencies, it was unclear whether OCEX can measure the development of proficiency during residency. This study evaluated whether OCEX can discriminate skill levels across years in training.
Methods: In 2017, the authors performed a retrospective analysis on modified OCEX evaluations collected for 22 residents over 3 years at 2 residency programs. OCEX subcomponent scores were averaged to generate a mean score for each evaluation, followed by linear regression analysis for mean scores over time for individual residents. One-way ANOVA with repeated measures was used to compare scores aggregated over an academic year between resident classes. The authors also surveyed internal faculty evaluators and nationwide ophthalmology residency program directors on their use of OCEX.
Results: Mean OCEX scores for individual residents and resident class averages showed variable trajectories over the course of residency. There was no consistent effect of increasing level of training on scores. Surveys of evaluators and program directors indicated different interpretations of the grading scale anchors and irregular participation by faculty.
Conclusion: This dataset suggests that, despite clear behavioral anchors and faculty development on the use of the tool, evaluators still apply inconsistent grading standards that limit the OCEX from accurately monitoring longitudinal development of resident clinical performance in real-world practice.