GDNF in Parkinson's disease: the perils of post-hoc power

J Neurosci Methods. 2007 Jul 30;163(2):193-6. doi: 10.1016/j.jneumeth.2007.05.003. Epub 2007 May 7.

Abstract

The practice of performing post-hoc power calculations for studies that do not demonstrate statistically significant results has been widely recognized in the scientific literature as being unhelpful and potentially misleading. However, this practice continues to cause confusion in the interpretation of results from clinical trials and other studies. Here, we examine the re-interpretation of a recent randomized, double-blind, placebo-controlled study of intraputamenally administered GDNF in late-stage Parkinson's disease patients [Hutchinson M, Gurney S, Newson R. GDNF in Parkinson disease: an object lesson in the tyranny of type II. J Neurosci Methods 2007;163:190-2]. Their main criticism is that the study was not large enough to detect clinically worthwhile effects and that the observed non-significant result does not contradict the promising results observed in two previous, small, open-label studies. We have carefully assessed the re-analysis of the data performed by Hutchinson et al. and found their conclusions to be flawed, in part because they are based on post-hoc power calculations. We have reaffirmed that the confidence interval for the treatment effect in the placebo-controlled study of GDNF shows that the trial is capable of excluding effects of GDNF of the magnitudes that were observed in the open-label studies and that the conclusions drawn in the original paper remain scientifically sound.

MeSH terms

  • Bias
  • Data Interpretation, Statistical
  • Glial Cell Line-Derived Neurotrophic Factor / therapeutic use*
  • Humans
  • Parkinson Disease / drug therapy*
  • Parkinson Disease / epidemiology*
  • Placebo Effect
  • Randomized Controlled Trials as Topic / standards
  • Randomized Controlled Trials as Topic / statistics & numerical data*
  • Reproducibility of Results
  • Sample Size
  • Selection Bias
  • Treatment Failure

Substances

  • Glial Cell Line-Derived Neurotrophic Factor