Using Learning Analytics to Examine Differences in Assessment Forms From Continuous Versus Episodic Supervisors of Family Medicine Residents

J Grad Med Educ. 2022 Oct;14(5):606-612. doi: 10.4300/JGME-D-21-00832.1.

Abstract

Background: It is assumed that there is a need for continuity of supervision within competency-based medical education, despite most evidence coming from the undergraduate medical education rather than the graduate medical education (GME) context. This evidence gap must be addressed to justify the time and effort needed to redesign GME programs to support continuity of supervision.

Objective: To examine differences in assessment behaviors of continuous supervisors (CS) versus episodic supervisors (ES), using completed formative assessment forms, FieldNotes, as a proxy.

Methods: The FieldNotes CS- and ES-entered for family medicine residents (N=186) across 3 outpatient teaching sites over 3 academic years (2015-2016, 2016-2017, 2017-2018) were examined using 2-sample proportion z-tests to determine differences on 3 FieldNote elements: competency (Sentinel Habit [SH]), Clinical Domain (CD), and Progress Level (PL).

Results: Sixty-nine percent (6104 of 8909) of total FieldNotes were analyzed. Higher proportions of CS-entered FieldNotes indicated SH3 (Managing patients with best practices), z=-3.631, P<.0001; CD2 (Care of adults), z=-8.659, P<.0001; CD3 (Care of the elderly), z=-4.592, P<.0001; and PL3 (Carry on, got it), z=-4.482, P<.0001. Higher proportions of ES-entered FieldNotes indicated SH7 (Communication skills), z=4.268, P<.0001; SH8 (Helping others learn), z=20.136, P<.0001; CD1 (Doctor-patient relationship/ethics), z=14.888, P<.0001; CD9 (Not applicable), z=7.180, P<.0001; and PL2 (In progress), z=5.117, P<.0001.

Conclusions: The type of supervisory relationship impacts assessment: there is variability in which competencies are paid attention to, which contexts or populations are included, and which progress levels are chosen.

MeSH terms

  • Adult
  • Aged
  • Clinical Competence
  • Competency-Based Education
  • Education, Medical, Graduate
  • Family Practice*
  • Humans
  • Internship and Residency*
  • Physician-Patient Relations

Grants and funding

Funding: This work was funded by the Social Sciences and Humanities Research Council of Canada.