Objective: Computerized neuropsychological assessment of concussion has rapidly expanded and Immediate Post-Concussion and Cognitive Testing (ImPACT) is among the most commonly used measures in this domain. ImPACT was primarily developed for use with athletic populations but continues to expand beyond athletics to settings such as the workplace and schools where motivational dispositions may vary. The purpose of the present study was to conduct a systematic review of existing research investigating the prevalence of invalid baseline results and the effectiveness of ImPACT's embedded invalidity indicators in detecting suspect effort.
Method: Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were followed in order to systematically structure a search across four databases and analysis of studies that presented data related to the prevalence of invalid performance and/or the effectiveness of ImPACT's embedded invalidity indicators.
Results: A total of 17 studies included prevalence rates of invalid performances or examined the effectiveness of ImPACT's invalidity indicators. Of the 17 studies, 12 included prevalence rates of invalid baseline results; and across this group of studies (after removing an outlier), the weighted prevalence rate of invalid baseline results was 6%. Four of the 17 studies examined the effectiveness of ImPACT's embedded invalidity indicators. ImPACT's embedded invalidity indicators correctly identified suboptimal effort in approximately 80% of individuals instructed to perform poorly and avoid detection ('coached') or instructed to perform poorly ('naïve').
Conclusions: These findings raise a number of issues pertaining to the use of ImPACT. Invalid performance incidence may increase with large group versus individual administration, use in nonclinical settings, and among those with Attention Deficit-Hyperactivity Disorder or learning disability. Additionally, the older desktop version of ImPACT appears to be associated with a higher rate of invalid performances than the online version. Although ImPACT's embedded invalidity indicators detect invalid performance at a rate of 6% on average, known group validity studies suggest that these measures miss invalid performance approximately 20% of the time when individuals purposefully underperform.
Keywords: Concussion assessment; ImPACT; invalid performance; malingering.