Using virtual-reality simulation to assess performance in endobronchial ultrasound

Respiration. 2013;86(1):59-65. doi: 10.1159/000350428. Epub 2013 May 23.


Background: For optimal treatment of patients with non-small cell lung carcinoma, it is essential to have physicians with competence in endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA). EBUS training and certification requirements are under discussion and the establishment of basic competence should be based on an objective assessment of performance.

Objectives: The aims of this study were to design an evidence-based and credible EBUS certification based on a virtual-reality (VR) EBUS simulator test.

Methods: Twenty-two respiratory physicians were divided into 3 groups: experienced EBUS operators (group 1, n = 6), untrained novices (group 2, n = 8) and simulator-trained novices (group 3, n = 8). Each physician performed two standardized simulated EBUS-TBNA procedures. Simulator metrics with discriminatory ability were identified and reliability was explored. Finally, the contrasting-groups method was used to establish a pass/fail standard, and the consequences of this standard were explored.

Results: Successfully sampled lymph nodes and procedure time were the only simulator metrics that showed statistically significant differences of p = 0.047 and p = 0.002, respectively. The resulting quality score (QS, i.e. sampled lymph nodes per minute) showed an acceptable reliability and a generalizability coefficient of 0.67. Reliability of 0.8 could be obtained by testing in 4 procedures. Median QS was 0.24 (range 0.21-0.26) and 0.098 (range 0.04-0.21) for groups 1 and 2, respectively (p = 0.001). The resulting pass/fail standard was 0.19. Group 3 had a median posttraining QS of 0.11 (range 0-0.17). None of them met the pass/fail standard.

Conclusions: With careful design of standardized tests, a credible standard setting and appropriate transfer studies, VR simulators could be an important first line in credentialing before proceeding to supervised performance on patients.

MeSH terms

  • Biopsy, Needle*
  • Bronchoscopy*
  • Carcinoma, Non-Small-Cell Lung / pathology*
  • Clinical Competence
  • Educational Measurement / methods*
  • Endosonography
  • Humans
  • Image-Guided Biopsy
  • Ultrasonography, Interventional*
  • User-Computer Interface*