Using Audience Response Systems for Real-Time Learning Assessments During Surgical Morbidity and Mortality Conference

Abstract

Objective: Morbidity and mortality (M&M) conference is a mainstay of surgical education. However, its effectiveness is poorly described. The purpose of this study was to demonstrate the feasibility of a real-time audience response system for learner assessment during M&M.

Design: We integrated a web-based audience response system into weekly M&M conference. First, this platform collected qualitative responses about the role of M&M. Then, we used the platform to direct questions to attendees in real time. Questions focused on surgical risk estimation and classifying root causes. Responses were grouped by training and compared to a validated risk tool's prediction. Root cause assignment concordance was statistically compared using Cohen's kappa between the pluralities of faculty responses to that of trainees.

Setting: General surgical residency program based at a tertiary academic medical center.

Participants: Affiliated categorical residents, preliminary residents, and clinical fellows.

Results: We enrolled 110 participants (38 faculty, 31 senior trainees, and 41 trainees). The majority of respondents (75.9%) cited education as the purpose of M&M, and all of respondents stated education as their personal motivation. Audience response questions were integrated into 34 unique case presentations. Mean absolute differences between predicted complication rates and attendees' predictions were highest for faculty (-9.4%, p = 0.009) and lowest for junior residents (-1.8%, p = 0.385). When assigning root cause of each morbidity, concordance between faculty and trainees was low to moderate (K = 0.41).

Conclusions: Assessment of learning during M&M can be performed in real time with discrimination observed by learner experience level. These data support development of this response platform to trend learner performance over time and to monitor targeted educational interventions at future M&Ms.

Keywords: Educational technology; Learning; Morbidity and mortality conference; Practice-Based Learning and Improvement; Professional education; Surgical education.