Statistical controversies in clinical research: statistical significance-too much of a good thing …

Ann Oncol. 2016 May;27(5):760-2. doi: 10.1093/annonc/mdw047. Epub 2016 Feb 9.


The use and interpretation of P values is a matter of debate in applied research. We argue that P values are useful as a pragmatic guide to interpret the results of a clinical trial, not as a strict binary boundary that separates real treatment effects from lack thereof. We illustrate our point using the result of BOLERO-1, a randomized, double-blind trial evaluating the efficacy and safety of adding everolimus to trastuzumab and paclitaxel as first-line therapy for HER2+ advanced breast cancer. In this trial, the benefit of everolimus was seen only in the predefined subset of patients with hormone receptor-negative breast cancer at baseline (progression-free survival hazard ratio = 0.66, P = 0.0049). A strict interpretation of this finding, based on complex 'alpha splitting' rules to assess statistical significance, led to the conclusion that the benefit of everolimus was not statistically significant either overall or in the subset. We contend that this interpretation does not do justice to the data, and we argue that the benefit of everolimus in hormone receptor-negative breast cancer is both statistically compelling and clinically relevant.

Trial registration: NCT00876395.

Keywords: P value; advanced breast cancer; everolimus; hormone receptor-negative; multiple comparisons; statistical significance.

Publication types

  • Review
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Breast Neoplasms / drug therapy
  • Breast Neoplasms / epidemiology*
  • Breast Neoplasms / pathology
  • Clinical Medicine / statistics & numerical data*
  • Double-Blind Method
  • Everolimus / therapeutic use
  • Female
  • Humans
  • Paclitaxel / therapeutic use
  • Proportional Hazards Models
  • Randomized Controlled Trials as Topic / statistics & numerical data*
  • Receptor, ErbB-2
  • Trastuzumab / therapeutic use


  • Everolimus
  • Receptor, ErbB-2
  • Trastuzumab
  • Paclitaxel

Associated data