Statistical controversies in clinical research: futility analyses in oncology-lessons on potential pitfalls from a randomized controlled trial

Abstract

Background: Pre-planned futility analyses are commonly used in oncology studies. The LUME-Lung 2 study (NCT00806819; 1199.14) was stopped early based on a pre-planned, non-binding futility analysis of investigator-assessed progression-free survival (PFS), although subsequent analysis showed that the primary endpoint of improvement in centrally reviewed PFS was met. Retrospective analyses were conducted to understand the discrepancy between interim futility and final analyses.

Materials and methods: LUME-Lung 2 investigated nintedanib in combination with pemetrexed versus placebo‒pemetrexed for the treatment of patients with advanced or recurrent non-squamous non-small cell lung cancer who had relapsed or failed one prior line of chemotherapy. Pre-planned futility analysis was carried out by the Data Monitoring Committee (DMC) after 50% of the events for the primary PFS analysis (713 events) had occurred; the threshold for futility was a conditional power of < 20%. Conditional/predictive powers and hazard ratios were calculated retrospectively after varying percentages of events had occurred for both investigator- and centrally reviewed PFS.

Results: At the time of the pre-planned futility analysis, the conditional power was 10.3% and the predictive power was 18.5%; no safety issues were identified. Retrospective analysis showed that the conditional and predictive powers fluctuated considerably over time for both investigator- and centrally reviewed PFS and that the power only dropped by a notable amount, and below the futility threshold, at the time of the futility analysis.

Conclusions: Retrospective investigations suggest that, had the DMC analysis been carried out at another time point, or had centrally reviewed PFS data been used, the futility outcome may have been different and the trial may have been continued. The design of futility analyses requires careful consideration and confirming negative futility outcomes by second analysis may be appropriate.

Trial number: NCT00806819.

Keywords: futility analysis; methodology; modelling, biostatistics; non-small cell lung cancer.