Reliable and Valid Assessment of Competence in Endoscopic Ultrasonography and Fine-Needle Aspiration for Mediastinal Staging of Non-Small Cell Lung Cancer

Endoscopy. 2012 Oct;44(10):928-33. doi: 10.1055/s-0032-1309892. Epub 2012 Jul 23.

Abstract

Background and study aims: Fine-needle aspiration (FNA) guided by endoscopic ultrasonography (EUS) is important in mediastinal staging of non-small cell lung cancer (NSCLC). Training standards and implementation strategies of this technique are currently under discussion. The aim of this study was to explore the reliability and validity of a newly developed EUS Assessment Tool (EUSAT) designed to measure competence in EUS - FNA for mediastinal staging of NSCLC.

Patients and methods: A total of 30 patients with proven or suspected NSCLC underwent EUS - FNA for mediastinal staging by three trainees and three experienced physicians. Their performances were assessed prospectively by three experts in EUS under direct observation and again 2 months later in a blinded fashion using digital video-recordings. Based on the assessments, intra-rater reliability, inter-rater reliability, and construct validity were explored.

Results: The intra-rater reliability was good (Cronbach's α = 0.80), but comparison of results based on direct observations and blinded video-recordings indicated a significant bias favoring consultants (P = 0.022). Inter-rater reliability was very good (Cronbach's α = 0.93). However, one rater assessing five procedures or two raters each assessing four procedures were necessary to secure a generalizability coefficient of 0.80. The assessment tool demonstrated construct validity by discriminating between trainees and experienced physicians (P = 0.034).

Conclusions: Competency in mediastinal staging of NSCLC using EUS and EUS - FNA can be assessed in a reliable and valid way using the EUSAT assessment tool. Measuring and defining competency and training requirements could improve EUS quality and benefit patient care.

Publication types

  • Validation Study

MeSH terms

  • Analysis of Variance
  • Biopsy, Needle*
  • Carcinoma, Non-Small-Cell Lung / diagnostic imaging
  • Carcinoma, Non-Small-Cell Lung / pathology*
  • Clinical Competence*
  • Endosonography*
  • Female
  • Humans
  • Lung Neoplasms / diagnostic imaging
  • Lung Neoplasms / pathology*
  • Male
  • Mediastinum / diagnostic imaging
  • Mediastinum / pathology*
  • Neoplasm Staging
  • Prospective Studies
  • Reproducibility of Results
  • Video Recording