The MISTELS program to measure technical skill in laparoscopic surgery : evidence for reliability

Surg Endosc. 2006 May;20(5):744-7. doi: 10.1007/s00464-005-3008-y. Epub 2006 Feb 27.


Background: The McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) is a series of five tasks with an objective scoring system. The purpose of this study was to estimate the interrater and test-retest reliability of the MISTELS metrics and to assess their internal consistency.

Methods: To determine interrater reliability, two trained observers scored 10 subjects, either live or on tape. Test-retest reliability was assessed by having 12 subjects perform two tests, the second immediately following the first. Interrater and test-retest reliability were assessed using intraclass correlation coefficients. Internal consistency between tasks was estimated using Cronbach's alpha.

Results: The interrater and test-retest reliabilities for the total scores were both excellent at 0.998 [95% confidence interval (CI), 0.985-1.00] and 0.892 (95% CI, 0.665-0.968), respectively. Cronbach's alpha for the first assessment of the test-retest was 0.86.

Conclusions: The MISTELS metrics have excellent reliability, which exceeds the threshold level of 0.8 required for high-stakes evaluations. These findings support the use of MISTELS for evaluation in many different settings, including residency training programs.

Publication types

  • Evaluation Study
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Clinical Competence*
  • Educational Measurement / methods*
  • Educational Measurement / standards
  • General Surgery / education*
  • Humans
  • Laparoscopy*
  • Observer Variation
  • Reproducibility of Results
  • Suture Techniques
  • Task Performance and Analysis
  • Teaching Materials*