Of apples and oranges, file drawers and garbage: why validity issues in meta-analysis will not go away

Clin Psychol Rev. 1997 Dec;17(8):881-901. doi: 10.1016/s0272-7358(97)00056-1.


This paper examines how threats to the validity of meta-analysis have been dealt with by clinical researchers employing this approach to literature review. Three validity threats were identified--mixing of dissimilar studies, publication bias, and inclusion of poor quality studies. Approaches to addressing these threats were evaluated for their effectiveness and popularity by surveying 32 published meta-analyses in clinical psychology. Distrust of meta-analysis, however, was found to transcend these validity threats. Other explanations for why this popular research strategy continues to receive widespread criticism were considered. Suggestions were made for how meta-analysis might better address these concerns.

Publication types

  • Research Support, Non-U.S. Gov't
  • Review

MeSH terms

  • Humans
  • Meta-Analysis as Topic*
  • Reproducibility of Results*
  • Research / standards*