Background: The global impression of a clinician is an Food and Drug Administration--mandated primary outcome measure for clinical trials in dementia. Reliability and validity of these measures are not well established.
Methods: A videotaped version of the Clinician's Interview Based Impression of Change (CIBIC) was evaluated. Raters were informed that the videotaped interviews were taken at baseline and 6 to 12 months later, when in fact half of the interviews were shown in reverse order. Ratings on "true order" interviews were compared with ratings on "reverse order" interviews. In addition, ratings by neurologists experienced in dementia were compared with those of less experienced raters.
Results: Inter-rater reliability of the neurologists was poor when measured by absolute agreement on a 7-point scale (kappa = 0.18). With a less stringent 3-point scale (better, worse, or unchanged), inter-rater reliability was significantly better for the true order videos (kappa = 0.51) than for the reversed order videos (kappa = 0.12). Validity also was reduced in the reverse order group: neurologists rated 90% of subjects correctly in the "true order" group and 63% correctly in the "reversed order" group. The inter-rater reliability of the neurologists was greater than the less experienced raters, but the validity of the neurologists' ratings was only marginally better.
Conclusions: The reliability and validity of the videotape CIBIC are reasonable when patients follow the expected course of gradual decline, but are poor when patients appear to improve. These findings suggest that global assessments should be modified as outcome measures in clinical trials with patients with dementia.