Flawed meta-analyses comparing psychotherapy with pharmacotherapy

Am J Psychiatry. 2000 Aug;157(8):1204-11. doi: 10.1176/appi.ajp.157.8.1204.


Objective: The author sought to illustrate the invalidity of meta-analyses that claim to quantitatively compare the benefits of psychotherapy to pharmacotherapy in patients with psychiatric disorders.

Method: Studies included in four meta-analyses were retrieved and their study designs evaluated.

Results: The meta-analyses compared effect sizes from disparate studies that were not uniformly blind, random, controlled, or of high quality. The studies did not directly address the comparative efficacy question or lacked assay sensitivity.

Conclusions: Numerous types of studies exemplify the need for caution in evaluating meta-analytic conclusions without first critically examining the included studies. Estimates of the relative efficacy of different treatments are not well founded when based almost exclusively on indirect, multiply confounded comparisons. Meta-analyses based on flawed studies or studies that lack demonstrated assay sensitivity are also inadequate for the criticism of treatment guidelines. Some bodies of data are inadequate to support a proper meta-analysis.

Publication types

  • Comparative Study
  • Research Support, U.S. Gov't, P.H.S.
  • Review

MeSH terms

  • Antidepressive Agents / therapeutic use
  • Clinical Trials as Topic
  • Cognitive Behavioral Therapy
  • Humans
  • Mental Disorders / drug therapy
  • Mental Disorders / therapy*
  • Meta-Analysis as Topic*
  • Panic Disorder / drug therapy
  • Panic Disorder / therapy
  • Psychotherapy*
  • Psychotropic Drugs / therapeutic use*
  • Reproducibility of Results
  • Research Design / standards


  • Antidepressive Agents
  • Psychotropic Drugs