Time crunch: increasing the efficiency of assessment of technical surgical skill via brief video clips

Abstract

Background: Video review for assessment of surgical performance is gaining popularity but is time consuming for busy expert reviewers, making review delays inevitable. Decreasing review time and including nonexpert reviewers may facilitate more timely reviews. We hypothesized that a shorter duration video clip would not affect the quality of expert ratings compared with full-length review. A secondary aim was to examine the reliability between expert and novice raters and how it was affected by video clip duration.

Methods: Videos of laparoscopic suturing performed on a live porcine model by premedical students, surgery residents, and fully trained surgeons were edited into 3 different durations: full, part, and 30-second versions. Video clips (n = 36) were reviewed by experienced surgeons (n = 3) and novice volunteers (n = 4) using the validated Global Operative Assessment of Laparoscopic Skills (GOALS) assessment. Videos were assigned randomly during 3 review cycles held 1 week apart. Each cycle included 1 iteration of the 12 performance videos. We assessed the impact of the duration of the video on reviewer scores and inter-rater reliability.

Results: Variance in scores for 2 of 4 GOALS domains was dependent on the duration of the video clip (P < .05). Total GOALS scores were greater for part and 30-second clips compared with full clips (P < .05). Inter-rater reliability was greatest for full clips (intraclass correlation = 0.68) and decreased significantly for shorter clips.

Conclusion: Our hypothesis was rejected as shorter video durations for surgical performance assessment led to inflated reviewer ratings both for expert and novice reviewers. Shortening duration of the video cannot be recommended for accurate performance assessment.