Observer agreement in the grading of oral epithelial dysplasia

Community Dent Oral Epidemiol. 2003 Aug;31(4):300-5. doi: 10.1034/j.1600-0528.2003.00013.x.


Objectives: To determine the extent of observer agreement in diagnosis of oral epithelial dysplasia (OED). Published studies of OED examiner agreement report relatively low agreement levels; however, these studies were limited by the methodologies employed.

Methods: For this study, 64 slides were each independently examined twice by three oral pathologists. Consistency was assessed by determining intra- and interexaminer agreement. Conformity was assessed by using the modal diagnosis as a gold standard.

Results: The group showed moderate interobserver agreement when grading the presence or absence of OED with a group-simple kappa (Ks) of 0.51 (95% CI = 0.42-0.61), and substantial agreement when using a 5-point ordinal scale with a group-weighted kappa (Kw) of 0.74 (95% CI = 0.64-0.85). The group showed fair to substantial intraexaminer agreement when assessing the presence or absence of OED, with Ks ranging from 0.22 to 0.78, and showing almost a perfect agreement using a 5-point ordinal scale, with Kw ranging from 0.82-0.96. Conformity with the comparison standard modal diagnosis was almost perfect, with pairwise Kw ranging from 0.81 to 0.92.

Conclusion: Overall, there was substantial intra- and interobserver consistency and almost perfect conformity in the grading of OED. Appropriate statistical methods are necessary to determine the degree of observer agreement.

MeSH terms

  • Adult
  • Aged
  • Aged, 80 and over
  • Carcinoma in Situ / diagnosis
  • Carcinoma in Situ / pathology
  • Carcinoma, Squamous Cell / diagnosis*
  • Carcinoma, Squamous Cell / pathology
  • Epithelium / pathology
  • Female
  • Humans
  • Male
  • Middle Aged
  • Mouth Mucosa / pathology
  • Mouth Neoplasms / diagnosis*
  • Mouth Neoplasms / pathology
  • Observer Variation
  • Precancerous Conditions / diagnosis*
  • Precancerous Conditions / pathology
  • Reproducibility of Results