Large-sample confidence intervals for validity and reliability coefficients

Psychol Methods. 2000 Sep;5(3):356-69. doi: 10.1037/1082-989x.5.3.356.

Abstract

Large-sample confidence intervals (CI) for reliability, validity, and unattenuated validity are presented. The CI for unattenuated validity is based on the Bonferroni inequality, which relies on one CI for test-retest reliability and one for validity. Covered are four reliability-validity situations: (a) both estimates were from random samples; (b) reliability was from a random sample but validity was from a selected sample; (c) validity was from a random sample but reliability was from a selected sample; and (d) both estimates were from selected samples. All CIs were evaluated by using a simulation. CIs on reliability, validity, or unattenuated validity are accurate as long as selection ratio is at least 20% and selected sample size is 100 or larger. When selection ratio is less than 20%, estimators tend to underestimate their parameters.

MeSH terms

  • Confidence Intervals*
  • Humans
  • Monte Carlo Method
  • Psychometrics*
  • Reproducibility of Results*
  • Sample Size
  • Sampling Studies