Development of a new critical thinking test using item response theory

Psychol Assess. 2006 Mar;18(1):100-5. doi: 10.1037/1040-3590.18.1.100.


The authors describe the initial development of the Wagner Assessment Test (WAT), an instrument designed to assess critical thinking, using the 5-faceted view popularized by the Watson-Glaser Critical Thinking Appraisal (WGCTA; G. B. Watson & E. M. Glaser, 1980). The WAT was designed to reduce the degree of successful guessing relative to the WGCTA by increasing the number of response alternatives (i.e., 80% of WGCTA items are 2-alternative, multiple-choice), a change that was hypothesized to result in more desirable test information and standard-error functions. Analyses using the 3-parameter logistic item response theory (IRT) model in a sample of undergraduates (N = 407) supported this prediction, even when the WAT item pool was shortened to match the length of the WGCTA. Convergent validity between full-pool IRT score estimates was r = .69. Implications for subsequent research on IRT-based measurement of critical thinking are discussed.

MeSH terms

  • Adolescent
  • Adult
  • Attitude*
  • Energy Intake
  • Female
  • Humans
  • Male
  • Models, Psychological
  • Psychological Theory*
  • Thinking*