Simpson's paradox visualized: the example of the rosiglitazone meta-analysis

BMC Med Res Methodol. 2008 May 30:8:34. doi: 10.1186/1471-2288-8-34.

Abstract

Background: Simpson's paradox is sometimes referred to in the areas of epidemiology and clinical research. It can also be found in meta-analysis of randomized clinical trials. However, though readers are able to recalculate examples from hypothetical as well as real data, they may have problems to easily figure where it emerges from.

Method: First, two kinds of plots are proposed to illustrate the phenomenon graphically, a scatter plot and a line graph. Subsequently, these can be overlaid, resulting in a overlay plot. The plots are applied to the recent large meta-analysis of adverse effects of rosiglitazone on myocardial infarction and to an example from the literature. A large set of meta-analyses is screened for further examples.

Results: As noted earlier by others, occurrence of Simpson's paradox in the meta-analytic setting, if present, is associated with imbalance of treatment arm size. This is well illustrated by the proposed plots. The rosiglitazone meta-analysis shows an effect reversion if all trials are pooled. In a sample of 157 meta-analyses, nine showed an effect reversion after pooling, though non-significant in all cases.

Conclusion: The plots give insight on how the imbalance of trial arm size works as a confounder, thus producing Simpson's paradox. Readers can see why meta-analytic methods must be used and what is wrong with simple pooling.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Child
  • Data Interpretation, Statistical*
  • Electric Power Supplies / adverse effects
  • Humans
  • Leukemia / etiology
  • Meta-Analysis as Topic*
  • Myocardial Infarction / chemically induced*
  • Randomized Controlled Trials as Topic
  • Rosiglitazone
  • Sample Size
  • Statistics as Topic
  • Thiazolidinediones / adverse effects*

Substances

  • Thiazolidinediones
  • Rosiglitazone