Inter-rater concordance and operating definitions of radiologic nodal feature assessment in human papillomavirus-positive oropharyngeal carcinoma

Oral Oncol. 2022 Feb:125:105716. doi: 10.1016/j.oraloncology.2022.105716. Epub 2022 Jan 14.

Abstract

Background and purpose: This study aims to evaluate the reliability of radiologic nodal feature assessment in clinical node-positive human papillomavirus-positive oropharyngeal carcinoma.

Materials and methods: Baseline CTs or MRIs of clinical node-positive human papillomavirus-positive oropharyngeal carcinoma diagnosed between 2012 and 2015 were reviewed independently by two neuroradiologists for seven nodal features: radiologic nodal involvement, cystic change, presence of necrosis, clustering, conglomeration, coalescence, and extranodal extension. Consensus operating definitions were derived after discussion. The features were re-reviewed in a randomly selected cohort. Levels of certainty (probability of presence: <25%, ∼50%, ∼75%, and >90%) were recorded. Interrater concordance was calculated using Cohen's kappa coefficient.

Results: A total of 413 patients (826 necks) were eligible. At initial review, the inter-rater kappa values for: radiologic nodal involvement, cystic change, necrosis, clustering, conglomeration, coalescence, and extranodal extension were 0.92, 0.64, 0.48, 0.32, 0.32, 0.62, and 0.56, respectively. A re-review of 94 randomly selected cases (188 necks) after consolidation of operating definitions for nodal features showed that the inter-rater kappa values of these features were 0.83, 0.62, 0.58, 0.32, 0.18, 0.68, and 0.74 when considering ≥50% certainty as positive, and improved to 0.94, 0.66, 0.59, 0.33, 0.19, 0.76, and 0.86 when considering ≥75% certainty as positive.

Conclusion: Clearly defined nomenclature results in improved interrater reliability when assessing radiologic nodal features, especially for coalescent adenopathy and extranodal extension. Higher levels of certainty are associated with higher inter-rater agreement. Radiology reporting should include clear definitions of clinically relevant nodal features as well as levels of certainty to serve various needs in clinical care and research.

Keywords: CT; Extranodal extension; Human papillomavirus; Inter-rater concordance; Lymph node; MRI; Oropharyngeal carcinoma; Radiologic nodal feature.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Alphapapillomavirus*
  • Carcinoma*
  • Extranodal Extension
  • Humans
  • Necrosis
  • Oropharyngeal Neoplasms* / diagnosis
  • Papillomavirus Infections* / complications
  • Papillomavirus Infections* / diagnostic imaging
  • Papillomavirus Infections* / pathology
  • Reproducibility of Results