Validation of video-based skill assessment in carotid artery stenting

Eur J Vasc Endovasc Surg. 2009 Jul;38(1):1-9. doi: 10.1016/j.ejvs.2009.03.008. Epub 2009 Apr 8.


Objectives: To develop weighted error-based, generic and procedure-specific rating scales, to validate these scales for video-based assessment during virtual carotid artery stent (CAS) procedures and correlate them with simulator-derived metrics.

Methods: A questionnaire was developed to assess the technique during live CAS procedures. Errors were rated from 1 (unimportant) to 5 (life-threatening) by 28 highly experienced CAS (>50 CAS) physicians. Virtual CAS procedure was performed by 21 interventionalists with varied CAS experience. Fluoroscopy screen and hand movements were video-taped, and simulator-derived metrics recorded. Experienced CAS practitioners then rated the video-taped performances using weighted error, generic and procedure-specific rating scales.

Results: Of the 23 errors assessed, 12 were regarded as moderate (score 3), six serious (score 4) and four life-threatening (score 5). The generic rating scale was able to detect significant differences in performance between inexperienced and experienced CAS operators (score 25 vs. 32 respectively, P<0.01). All scoring systems demonstrated good inter-rater reliability (alpha=0.61-0.87). Significant correlations were observed between simulator-derived and video-based scores: weighted error-based score (r: 0.76, P<0.01), generic (r: 0.62, P<0.01) and procedure-specific (r: 0.76, P<0.01) rating scales.

Conclusions: The generic endovascular rating scale differentiated between levels of CAS experience among skilled interventionalists and correlated to simulator-based error scoring.

Publication types

  • Comparative Study
  • Research Support, Non-U.S. Gov't
  • Validation Study

MeSH terms

  • Blood Vessel Prosthesis Implantation / standards*
  • Carotid Stenosis / surgery*
  • Clinical Competence / standards*
  • Humans
  • Medical Errors / prevention & control*
  • Medical Errors / statistics & numerical data
  • Outcome Assessment, Health Care / methods*
  • Stents*
  • Surveys and Questionnaires
  • User-Computer Interface
  • Video Recording / methods*