Objective: The purpose of the study was to evaluate the inter-judge and intra-judge reliability of raters using the Voice-Vibratory Assessment with Laryngeal Imaging (VALI) rating form that was developed for assessing videostroboscopy and high-speed videoendoscopic (HSV) recordings.
Subjects and methods: Nine speech-language pathologists with an average of 12.8 years of experience with laryngeal imaging were trained to use the VALI form for rating 66 de-identified and randomized samples with voice disorders. Inter-judge reliability for parameters with scale data (amplitude, mucosal wave, nonvibratory portion, supraglottic activity, phase closure, symmetry, and regularity or periodicity) was assessed with intraclass correlations, and parameters with nominal data (glottal closure, vertical level, and free edge contour) were assessed with Fleiss' kappa. Intra-judge reliability was assessed using the Spearman rho statistic for scale data and percentage of concordant pairs for nominal data.
Results: Inter-judge reliability for parameters with scale data ranged from 0.57 to 0.96 for stroboscopy and from 0.81 to 0.94 for HSV. For nominal parameters, correlations ranged from 0.18 to 0.35 for stroboscopy and from 0.13 to 0.33 for HSV. Intra-judge reliability correlations for parameters with scale data ranged from 0.19 to 0.87 for stroboscopy and from 0.28 to 0.85 for HSV. For parameters with nominal data, percentage of concordance ranged from 44% to 78% for stroboscopy and from 52% to 89% for HSV.
Conclusions: The VALI rating form and the training protocol is a first, a priori developed rating form that includes visual-perceptual ratings of both stroboscopy and HSV. The current form can be used to make reliable visual-perceptual judgments for selected features of vibratory motion from stroboscopy and HSV.
Keywords: Clinical judgments and training; High-speed videoendoscopy; Laryngeal imaging; Stroboscopy; Visual-perceptual assessment.
Copyright © 2017 The Voice Foundation. Published by Elsevier Inc. All rights reserved.