Long-Term Skills Retention Following a Randomized Prospective Trial on Adaptive Procedural Training

Abstract

Objective: Cumulative sum (CuSum) is a real-time proficiency-monitoring tool adapted for simulation-based training. This study's objective was to investigate long-term outcomes of a double blinded, randomized control trial conducted with medical students assessing CuSum-guided curriculum against volume-based standards. The trial found a nearly 20% reduction in practice time to reach proficiency using the CuSum curriculum but long-term effects of decreased practice volume on proficiency is unknown.

Design: Prior participants completed a survey assessing confidence, exposure, and feedback at 12 to 18 months following trial completion. They underwent retention testing of suturing, intubation, and central venous catheter placement (CVC), which was video-recorded and assessed by an expert evaluator. Baseline characteristics among repeat subjects were compared using chi-squared tests. Retention and initial trial outcome were compared using paired parametric statistical methods.

Setting: The study was conducted at a major tertiary care center and training hospital.

Participants: Medical students, which completed the initial randomized control trial were eligible for enrollment. A total of 30/46(65%) responded to the survey, whereas 33/46(72%) completed retention testing.

Results: Average scores and decay in procedural tasks over time for suturing, intubation and CVC were 91.6% (-4.7%), 86.1% (-4.1%), and 76.2% (-14.8%), respectively. Compared to the control group, the CuSum group mean difference in retention evaluation scores was -5.6% (p = 0.12). Confidence was not associated with initial or retention testing performance in any procedural task. Higher confidence was associated with additional exposure to the procedural task in suturing and intubation (p = 0.03 and p = 0.02, respectively). For intubation, higher confidence was reported by participants who received positive feedback (p = 0.01), and those assigned to the volume-based training arm (p = 0.03).

Conclusion: CuSum-guided training was equivalent to conventional training for suturing, intubation, and CVC. These findings importantly suggest medical students can retain competency in invasive surgical tasks with modest decay in proficiency over time regardless of initial training method.

Keywords: Practice-Based Learning and Improvement; competency-based curriculum; cumulative sum; medical student education; simulation training; surgical education.