The PROMIS of better outcome assessment: responsiveness, floor and ceiling effects, and Internet administration

J Rheumatol. 2011 Aug;38(8):1759-64. doi: 10.3899/jrheum.110402.

Abstract

Objective: Use of item response theory (IRT) and, subsequently, computerized adaptive testing (CAT), under the umbrella of the NIH-PROMIS initiative (National Institutes of Health-Patient-Reported Outcomes Measurement Information System), to bring strong new assets to the development of more sensitive, more widely applicable, and more efficiently administered patient-reported outcome (PRO) instruments. We present data on current progress in 3 crucial areas: floor and ceiling effects, responsiveness to change, and interactive computer-based administration over the Internet.

Methods: We examined nearly 1000 patients with rheumatoid arthritis and related diseases in a series of studies including a one-year longitudinal examination of detection of change; compared responsiveness of the Legacy SF-36 and HAQ-DI instruments with IRT-based instruments; performed a randomized head-to-head trial of 4 modes of item administration; and simulated the effect of lack of floor and ceiling items upon statistical power and sample sizes.

Results: IRT-based PROMIS instruments are more sensitive to change, resulting in the potential to reduce sample size requirements substantially by up to a factor of 4. The modes of administration tested did not differ from each other in any instance by more than one-tenth of a standard deviation. Floor and ceiling effects greatly reduce the number of available subjects, particularly at the ceiling.

Conclusion: Failure to adequately address floor and ceiling effects, which determine the range of an instrument, can result in suboptimal assessment of many patients. Improved items, improved instruments, and computer-based administration improve PRO assessment and represent a fundamental advance in clinical outcomes research.

Publication types

  • Research Support, N.I.H., Extramural

MeSH terms

  • Arthritis, Rheumatoid / physiopathology
  • Arthritis, Rheumatoid / therapy
  • Disability Evaluation
  • Humans
  • Internet*
  • National Institutes of Health (U.S.)
  • Outcome Assessment, Health Care / methods*
  • Patient Satisfaction*
  • Randomized Controlled Trials as Topic
  • Research Design
  • Surveys and Questionnaires
  • Treatment Outcome*
  • United States