Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program

Surg Endosc. 2011 Apr;25(4):1136-42. doi: 10.1007/s00464-010-1329-y. Epub 2010 Sep 11.

Abstract

Introduction: The National Training Program for laparoscopic colorectal surgery (LCS) provides supervised training to colorectal surgeons in England. The purpose of this study was to create, validate, and implement a method for monitoring training progression in laparoscopic colorectal surgery that met the requirements of a good assessment tool.

Methods: A generic scale for different tasks in LCS was created under the guidance of a national expert group. The scores were defined by the extent to which the trainees were dependent on support (1 = unable to perform, 5 = unaided (benchmark), 6 = proficient). Trainers were asked to rate their trainees after each supervised case; trainees completed a similar self-assessment form. Construct validity was evaluated comparing scores of trainees at different experience levels (1-5, 6-10, 11-15, 16+) using the Wilcoxon signed-rank test and ANOVA. Internal consistency was determined by Crohnbach's alpha, interrater reliability by comparing peer- and self-assessment (interclass correlation coefficient, ICC). Proficiency gain curves were plotted using CUSUM charts.

Results: Analysis included 610 assessments (333 by trainers and 277 by trainees). There was high interrater reliability (ICC = 0.867), internal consistency (α = 0.920), and construct validity [F(3,40) = 6.128, p < 0.001]. Detailed analysis of proficiency gain curves demonstrates that theater setup, exposure, and anastomosis were performed independently after 5 to 15 sessions, and the dissection of the vascular pedicle took 24 cases. Mobilization of the colon and of the splenic/hepatic flexure took more than 25 procedures. Median assessment time was 3.3 (interquartile range (IQR) 1-5) minutes and the tool was accepted as useful [median score 5 of 6 (IQR 4-5)].

Discussion: A valid and reliable monitoring tool for surgical training has been implemented successfully into the National Training Program. It provides a description of an individualized proficiency gain curve in terms of both the level of support required and the competency level achieved.

Publication types

  • Research Support, Non-U.S. Gov't
  • Validation Study

MeSH terms

  • Anastomosis, Surgical / education
  • Anastomosis, Surgical / methods
  • Anastomosis, Surgical / standards
  • Clinical Competence
  • Colonic Diseases / surgery
  • Digestive System Surgical Procedures / education*
  • Digestive System Surgical Procedures / methods
  • Digestive System Surgical Procedures / standards
  • Educational Measurement / methods*
  • Educational Measurement / statistics & numerical data
  • England
  • Humans
  • Laparoscopy / education*
  • Laparoscopy / methods
  • Laparoscopy / standards
  • Learning Curve
  • Motor Skills
  • Observer Variation
  • Peer Review
  • Psychomotor Performance*
  • Rectal Diseases / surgery
  • Reproducibility of Results
  • Self-Assessment