Effects of training in direct observation of medical residents' clinical competence: a randomized trial

Ann Intern Med. 2004 Jun 1;140(11):874-81. doi: 10.7326/0003-4819-140-11-200406010-00008.

Abstract

Background: Faculty observation of residents and students performing clinical skills is essential for reliable and valid evaluation of trainees.

Objective: To evaluate the efficacy of a new multifaceted method of faculty development called direct observation of competence training.

Design: Controlled trial of faculty from 16 internal medicine residency programs using a cluster randomization design.

Setting: Academic medical centers.

Participants: 40 internal medicine teaching faculty members: 17 in the intervention group and 23 in the control group.

Measurements: Changes in faculty comfort performing direct observation, faculty satisfaction with workshop, and changes in faculty rating behaviors 8 months after completing the training.

Intervention: The direct observation of competence workshop combines didactic mini-lectures, interactive small group and videotape evaluation exercises, and evaluation skill practice with standardized residents and patients.

Results: 37 faculty members (16 in the intervention group and 21 in the control group) completed the study. Most of the faculty in the intervention group (14 [88%]) reported that they felt significantly more comfortable performing direct observation compared with control group faculty (4 [19%]) (P = 0.04), and all intervention faculty rated the training as outstanding. For 9 videotaped clinical encounters, intervention group faculty were more stringent than controls in their evaluations of medical interviewing, physical examination, and counseling; differences in ratings for medical interviewing and physical examination remained statistically significant even after adjustment for baseline rating behavior.

Limitations: The study involved a limited number of residency programs, and faculty did not rate the performance of actual residents.

Conclusion: Direct observation of competence training, a new multifaceted approach to faculty development, leads to meaningful changes in rating behaviors and in faculty comfort with evaluation of clinical skills.

Publication types

  • Clinical Trial
  • Randomized Controlled Trial
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adult
  • Clinical Competence*
  • Faculty, Medical / standards*
  • Female
  • Humans
  • Inservice Training*
  • Internal Medicine / education*
  • Internship and Residency*
  • Male
  • Observation*
  • Videotape Recording