Operating Characteristics of a Rank Correlation Test for Publication Bias

Biometrics. 1994 Dec;50(4):1088-101.

Abstract

An adjusted rank correlation test is proposed as a technique for identifying publication bias in a meta-analysis, and its operating characteristics are evaluated via simulations. The test statistic is a direct statistical analogue of the popular "funnel-graph." The number of component studies in the meta-analysis, the nature of the selection mechanism, the range of variances of the effect size estimates, and the true underlying effect size are all observed to be influential in determining the power of the test. The test is fairly powerful for large meta-analyses with 75 component studies, but has only moderate power for meta-analyses with 25 component studies. However, in many of the configurations in which there is low power, there is also relatively little bias in the summary effect size estimate. Nonetheless, the test must be interpreted with caution in small meta-analyses. In particular, bias cannot be ruled out if the test is not significant. The proposed technique has potential utility as an exploratory tool for meta-analysts, as a formal procedure to complement the funnel-graph.

Publication types

  • Comparative Study
  • Research Support, U.S. Gov't, P.H.S.

MeSH terms

  • Bias*
  • Case-Control Studies
  • Chlamydia Infections / epidemiology
  • Chlorine
  • Contraceptives, Oral
  • Female
  • Humans
  • Lung Neoplasms / epidemiology
  • Mathematics
  • Meta-Analysis as Topic
  • Models, Statistical*
  • Neoplasms / epidemiology
  • Odds Ratio
  • Publishing*
  • Tobacco Smoke Pollution / adverse effects
  • Water Supply

Substances

  • Contraceptives, Oral
  • Tobacco Smoke Pollution
  • Chlorine