Models for estimating the number of unpublished studies

Stat Med. 1996 Dec 15;15(23):2493-507. doi: 10.1002/(SICI)1097-0258(19961215)15:23<2493::AID-SIM381>3.0.CO;2-C.

Abstract

The possible existence of unreported studies can cast doubt on the conclusions of a meta-analytic summary of the literature, particularly if there is reason to believe that there is a publication bias against non-significant results. The present article proposes two general models that describe how the preponderance of published studies could report significant p-values even when testing a null hypothesis that is, in fact, true. Each such model allows one to estimate the number, N, of unpublished studies using the p-values reported in the published studies; the meta-analyst can then evaluate the plausibility of this estimated value of N, or related confidence bounds. Use of models of the kind suggested here allows meta-analysts to assess the problem of unpublished studies from various perspectives and thus can lead to greater understanding of, and confidence in, meta-analytic conclusions.

Publication types

  • Research Support, U.S. Gov't, Non-P.H.S.
  • Research Support, U.S. Gov't, P.H.S.

MeSH terms

  • Bias
  • Confidence Intervals
  • Delivery of Health Care / statistics & numerical data
  • Educational Measurement / statistics & numerical data
  • Endocrinology
  • Intelligence Tests / statistics & numerical data
  • Likelihood Functions
  • Meta-Analysis as Topic*
  • Models, Statistical*
  • Publishing / statistics & numerical data*
  • Random Allocation