Reliability and construct validity of a structured technical skills assessment form

Am J Surg. 1994 Apr;167(4):423-7. doi: 10.1016/0002-9610(94)90128-7.


Current methods of evaluating technical competence of surgical residents are subjective and potentially unreliable. This study assesses the reliability and construct validity of a new format for the assessment of technical ability, the two part Structured Technical Skills Assessment Form (STSAF). Part I, which is completed while an operation is proceeding consists of approximately 120 essential components of the procedure. Part II, completed at the end of the operation, is a 10-point global rating form. Forty-one operations were evaluated using the STSAF, with multiple observers present at 26. Inter-rater reliability of both Parts I and II were high (.78 and .73, respectively). Statistically significant differences were noted between senior-resident and junior-resident performances, suggesting construct validity. The incorporation of structured guidelines to the assessment of technical skill leads to high inter-rater reliability and construct validity, which ultimately may result in improved and reproducible evaluations of surgical trainees.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Cholecystectomy
  • Clinical Competence*
  • Educational Measurement / methods
  • General Surgery / education*
  • Hernia, Inguinal / surgery
  • Humans
  • Internship and Residency*
  • Intestines / surgery
  • Observer Variation
  • Pilot Projects
  • Professional Competence*
  • Reproducibility of Results