Development of embedded performance validity indicators in the NIH Toolbox Cognitive Battery

Psychol Assess. 2021 Jan;33(1):90-96. doi: 10.1037/pas0000958. Epub 2020 Oct 29.


To assess noncredible performance on the NIH Toolbox Cognitive Battery (NIHTB-CB), we developed embedded validity indicators (EVIs). Data were collected from 98 adults (54.1% female) as part of a prospective multicenter cross-sectional study at 4 mild traumatic brain injury (mTBI) specialty clinics. Traditional EVIs and novel item-based EVIs were developed for the NIHTB-CB using the Medical Symptom Validity Test (MSVT) as criterion. The signal detection profile of individual EVIs varied greatly. Multivariate models had superior classification accuracy. Failing ≥4 traditional EVIs at the liberal cutoff or ≥3 at the conservative cutoff produced a good combination of sensitivity (.57 to .61) and specificity (.92 to .94) to MSVT. Combining the traditional and item-based EVIs improved sensitivity (.65 to .70) at comparable specificity (.91 to .95). In conclusion, newly developed EVIs within the NIHTB-CB effectively discriminated between patients who passed versus failed the MSVT. Aggregating EVIs within the same category into validity composites improved signal detection over univariate cutoffs. Item-based EVIs improved classification accuracy over that of traditional EVIs. However, the marginal gains hardly justify the burden of extra calculations. The newly introduced EVIs require cross-validation before wide-spread research or clinical application. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Publication types

  • Clinical Trial
  • Multicenter Study

MeSH terms

  • Adolescent
  • Adult
  • Aged
  • Brain Concussion / diagnosis*
  • Cognition*
  • Cross-Sectional Studies
  • Female
  • Humans
  • Male
  • Middle Aged
  • Multivariate Analysis
  • Neuropsychological Tests*
  • Prospective Studies
  • Psychometrics
  • Reproducibility of Results
  • Sensitivity and Specificity
  • Young Adult