Visual and region of interest-based inter-rater agreement in the assessment of the diffusion-weighted imaging- fluid-attenuated inversion recovery mismatch

Stroke. 2014 Apr;45(4):1170-2. doi: 10.1161/STROKEAHA.113.002661. Epub 2014 Feb 20.

Abstract

Background and purpose: WAKE-UP is a randomized, placebo-controlled MRI-based trial of thrombolysis in wake-up stroke using the mismatch between a lesion's visibility in diffusion-weighted imaging and fluid-attenuated inversion recovery (FLAIR) sequences as its main imaging inclusion criterion. Visual judgment of lesion conspicuity on FLAIR is however methodically limited by moderate inter-rater agreement. We therefore sought to improve rating homogeneity by incorporating quantitative signal intensity measurements.

Methods: One hundred forty-three data sets of patients with acute ischemic stroke were visually rated by 8 raters with respect to WAKE-UP study inclusion and exclusion criteria, and inter-rater agreement was calculated. A subanalysis was performed on 45 cases to determine a threshold value of relative signal intensity (rSI) between the ischemic lesion and contralateral healthy tissue which best corresponded to a visually established verdict of FLAIR positivity. The usefulness of this threshold in improving inter-rater agreement was evaluated in an additional sample of 50 patients.

Results: Inter-rater agreement for inclusion into the WAKE-UP trial was 73% with a free-marginal κ of 0.46. A threshold of rSI which best correlated with the visual rating of lesions as FLAIR positive was 1.20. The addition of rSI measurements to visual evaluation did not change the inter-rater agreement.

Conclusions: Introducing a semiquantitative measure for FLAIR rSI did not improve the agreement between individual raters. However, enhancing visual assessment with rSI measurements can provide reassurance to local investigators in cases of uncertainty.

Keywords: magnetic resonance imaging; stroke.

Publication types

  • Comparative Study
  • Observational Study
  • Randomized Controlled Trial
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Acute Disease
  • Aged
  • Brain Ischemia / pathology*
  • Diffusion Magnetic Resonance Imaging / methods*
  • Diffusion Magnetic Resonance Imaging / statistics & numerical data*
  • Female
  • Humans
  • Male
  • Middle Aged
  • Observer Variation
  • Recovery of Function
  • Stroke / pathology*
  • Thrombolytic Therapy / methods*
  • Visual Pathways / pathology*