Endobronchial ultrasound skills and tasks assessment tool: assessing the validity evidence for a test of endobronchial ultrasound-guided transbronchial needle aspiration operator skill

Am J Respir Crit Care Med. 2012 Oct 15;186(8):773-9. doi: 10.1164/rccm.201111-1968OC. Epub 2012 Jul 26.


Rationale: Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is becoming standard of care for the sampling of mediastinal adenopathy. The need for a safe, effective, accurate procedure makes EBUS-TBNA ideal for mastery training and testing.

Objectives: The Endobronchial Ultrasound Skills and Tasks Assessment Tool (EBUS-STAT) was created as an objective competency-oriented assessment tool of EBUS-TBNA skills and knowledge. This study demonstrates the reliability and validity evidence of this tool.

Methods: The EBUS-STAT objectively scores the EBUS-TBNA operator's skills, including atraumatic airway introduction and navigation, ultrasound image acquisition and optimization, identification of mediastinal nodal and vascular structures, EBUS-TBNA sampling, and recognition of EBUS/computed tomography images of mediastinal structures. It can be administered at the bedside or using combination of low- and high-fidelity simulation platforms. Two independent testers administered the EBUS-STAT to 24 operators at three levels of EBUS-TBNA experience (8 beginners, 8 intermediates, and 8 experienced) at three institutions; operators were also asked to self-assess their skills. Scores were analyzed for intertester reliability, correlation with prior EBUS-TBNA experience, and association with self-assessments.

Measurements and main results: Intertester reliability between testers was very high (r = 0.9991, P < 0.00005). Mean EBUS-STAT scores for beginner, intermediate, and experienced groups, respectively, were 31.1, 74.9, and 93.6 out of 100 (F(2,21) = 118.6, P < 0.0001). Groups were nonoverlapping: post hoc tests showed each group differed significantly from the others (P < 0.001). Self-assessments corresponded closely to actual EBUS-STAT scores (r(2) = 0.81, P < 0.001).

Conclusions: The EBUS-STAT can be used to reliably and objectively score and classify EBUS-TBNA operators from novice to expert. Its use to assess and document the acquisition of knowledge and skill is a step toward the goal of mastery training in EBUS-TBNA.

Publication types

  • Multicenter Study
  • Validation Study

MeSH terms

  • Benchmarking
  • Bronchoscopy / education*
  • Clinical Competence*
  • Educational Measurement*
  • Endoscopic Ultrasound-Guided Fine Needle Aspiration*
  • Humans
  • Multivariate Analysis
  • Observer Variation
  • Regression Analysis
  • Reproducibility of Results
  • Self-Assessment
  • United States