Validation of Computerized Adaptive Testing in an Outpatient Nonacademic Setting: The VOCATIONS Trial

Psychiatr Serv. 2015 Oct;66(10):1091-6. doi: 10.1176/ Epub 2015 Jun 1.


Objective: Computerized adaptive testing (CAT) provides an alternative to fixed-length assessments. The study validated a suite of computerized adaptive tests for mental health (CAT-MH) in a community psychiatric sample.

Methods: A total of 145 adults from a community outpatient clinic, including 19 with no history of a mental disorder (control group), were prospectively evaluated with CAT for depression (CAD-MDD and CAT-DI), mania (CAT-MANIA), and anxiety symptoms (CAT-ANX). Ratings were compared with gold-standard psychiatric assessments, including the Structured Clinical Interview for DSM-IV-TR (SCID), Hamilton Rating Scale for Depression (HAM-D-25), Patient Health Questionnaire (PHQ-9), Center for Epidemiologic Studies Depression Scale (CES-D), and Global Assessment of Functioning (GAF).

Results: Sensitivity and specificity for CAD-MDD were .96 and .64, respectively (.96 and 1.00 for major depression versus the control group). CAT for depression severity (CAT-DI) correlated well with the HAM-D-25 (r=.79), PHQ-9 (r=.90), and CES-D (r=.90) and had an odds ratio (OR) of 27.88 across its range for current SCID major depressive disorder. CAT-ANX correlated with the HAM-D-25 (r=.73), PHQ-9 (r=.78), and CES-D (r=.81) and had an OR of 11.52 across its range for current SCID generalized anxiety disorder. CAT-MANIA did not correlate well with the HAM-D-25 (r=.31), PHQ-9 (r=.37), and CES-D (r=.39), but it had an OR of 11.56 across its range for a current SCID bipolar diagnosis. Participants found the CAT-MH acceptable and easy to use, averaging 51.7 items and 9.4 minutes to complete the full battery.

Conclusions: Compared with gold-standard diagnostic and assessment measures, CAT-MH provided an effective, rapidly administered assessment of psychiatric symptoms.

Publication types

  • Research Support, N.I.H., Extramural
  • Research Support, Non-U.S. Gov't
  • Validation Study

MeSH terms

  • Adult
  • Anxiety Disorders / diagnosis*
  • Bipolar Disorder / diagnosis*
  • Cross-Sectional Studies
  • Depressive Disorder, Major / diagnosis*
  • Diagnosis, Computer-Assisted / standards*
  • Diagnostic and Statistical Manual of Mental Disorders
  • Female
  • Humans
  • Male
  • Middle Aged
  • Outpatients / psychology*
  • Prospective Studies
  • Psychiatric Status Rating Scales
  • Sensitivity and Specificity
  • Severity of Illness Index