Statistical controversies in clinical research: futility analyses in oncology-lessons on potential pitfalls from a randomized controlled trial

E. Lesaffre, M. J. Edelman, Nasser Hanna, K. Park, N. Thatcher, S. Willemsen, B. Gaschler-Markefski, R. Kaiser, C. Manegold

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Background: Pre-planned futility analyses are commonly used in oncology studies. The LUME-Lung 2 study (NCT00806819; 1199.14) was stopped early based on a pre-planned, non-binding futility analysis of investigator-assessed progression-free survival (PFS), although subsequent analysis showed that the primary endpoint of improvement in centrally reviewed PFS was met. Retrospective analyses were conducted to understand the discrepancy between interim futility and final analyses.

Materials and methods: LUME-Lung 2 investigated nintedanib in combination with pemetrexed versus placebo‒pemetrexed for the treatment of patients with advanced or recurrent non-squamous non-small cell lung cancer who had relapsed or failed one prior line of chemotherapy. Pre-planned futility analysis was carried out by the Data Monitoring Committee (DMC) after 50% of the events for the primary PFS analysis (713 events) had occurred; the threshold for futility was a conditional power of < 20%. Conditional/predictive powers and hazard ratios were calculated retrospectively after varying percentages of events had occurred for both investigator- and centrally reviewed PFS.

Results: At the time of the pre-planned futility analysis, the conditional power was 10.3% and the predictive power was 18.5%; no safety issues were identified. Retrospective analysis showed that the conditional and predictive powers fluctuated considerably over time for both investigator- and centrally reviewed PFS and that the power only dropped by a notable amount, and below the futility threshold, at the time of the futility analysis.

Conclusions: Retrospective investigations suggest that, had the DMC analysis been carried out at another time point, or had centrally reviewed PFS data been used, the futility outcome may have been different and the trial may have been continued. The design of futility analyses requires careful consideration and confirming negative futility outcomes by second analysis may be appropriate.

Trial number: NCT00806819.

Original languageEnglish (US)
Pages (from-to)1419-1426
Number of pages8
JournalAnnals of oncology : official journal of the European Society for Medical Oncology
Volume28
Issue number7
DOIs
StatePublished - Jul 1 2017

Fingerprint

Medical Futility
Randomized Controlled Trials
Disease-Free Survival
Research
Clinical Trials Data Monitoring Committees
Research Personnel
Pemetrexed
Lung
Survival Analysis
Non-Small Cell Lung Carcinoma

Keywords

  • futility analysis
  • methodology
  • modelling, biostatistics
  • non-small cell lung cancer

ASJC Scopus subject areas

  • Hematology
  • Oncology

Cite this

Statistical controversies in clinical research : futility analyses in oncology-lessons on potential pitfalls from a randomized controlled trial. / Lesaffre, E.; Edelman, M. J.; Hanna, Nasser; Park, K.; Thatcher, N.; Willemsen, S.; Gaschler-Markefski, B.; Kaiser, R.; Manegold, C.

In: Annals of oncology : official journal of the European Society for Medical Oncology, Vol. 28, No. 7, 01.07.2017, p. 1419-1426.

Research output: Contribution to journalArticle

Lesaffre, E. ; Edelman, M. J. ; Hanna, Nasser ; Park, K. ; Thatcher, N. ; Willemsen, S. ; Gaschler-Markefski, B. ; Kaiser, R. ; Manegold, C. / Statistical controversies in clinical research : futility analyses in oncology-lessons on potential pitfalls from a randomized controlled trial. In: Annals of oncology : official journal of the European Society for Medical Oncology. 2017 ; Vol. 28, No. 7. pp. 1419-1426.
@article{563c4f65e9f5483da9c87747085c699b,
title = "Statistical controversies in clinical research: futility analyses in oncology-lessons on potential pitfalls from a randomized controlled trial",
abstract = "Background: Pre-planned futility analyses are commonly used in oncology studies. The LUME-Lung 2 study (NCT00806819; 1199.14) was stopped early based on a pre-planned, non-binding futility analysis of investigator-assessed progression-free survival (PFS), although subsequent analysis showed that the primary endpoint of improvement in centrally reviewed PFS was met. Retrospective analyses were conducted to understand the discrepancy between interim futility and final analyses.Materials and methods: LUME-Lung 2 investigated nintedanib in combination with pemetrexed versus placebo‒pemetrexed for the treatment of patients with advanced or recurrent non-squamous non-small cell lung cancer who had relapsed or failed one prior line of chemotherapy. Pre-planned futility analysis was carried out by the Data Monitoring Committee (DMC) after 50{\%} of the events for the primary PFS analysis (713 events) had occurred; the threshold for futility was a conditional power of < 20{\%}. Conditional/predictive powers and hazard ratios were calculated retrospectively after varying percentages of events had occurred for both investigator- and centrally reviewed PFS.Results: At the time of the pre-planned futility analysis, the conditional power was 10.3{\%} and the predictive power was 18.5{\%}; no safety issues were identified. Retrospective analysis showed that the conditional and predictive powers fluctuated considerably over time for both investigator- and centrally reviewed PFS and that the power only dropped by a notable amount, and below the futility threshold, at the time of the futility analysis.Conclusions: Retrospective investigations suggest that, had the DMC analysis been carried out at another time point, or had centrally reviewed PFS data been used, the futility outcome may have been different and the trial may have been continued. The design of futility analyses requires careful consideration and confirming negative futility outcomes by second analysis may be appropriate.Trial number: NCT00806819.",
keywords = "futility analysis, methodology, modelling, biostatistics, non-small cell lung cancer",
author = "E. Lesaffre and Edelman, {M. J.} and Nasser Hanna and K. Park and N. Thatcher and S. Willemsen and B. Gaschler-Markefski and R. Kaiser and C. Manegold",
year = "2017",
month = "7",
day = "1",
doi = "10.1093/annonc/mdx042",
language = "English (US)",
volume = "28",
pages = "1419--1426",
journal = "Annals of Oncology",
issn = "0923-7534",
publisher = "Oxford University Press",
number = "7",

}

TY - JOUR

T1 - Statistical controversies in clinical research

T2 - futility analyses in oncology-lessons on potential pitfalls from a randomized controlled trial

AU - Lesaffre, E.

AU - Edelman, M. J.

AU - Hanna, Nasser

AU - Park, K.

AU - Thatcher, N.

AU - Willemsen, S.

AU - Gaschler-Markefski, B.

AU - Kaiser, R.

AU - Manegold, C.

PY - 2017/7/1

Y1 - 2017/7/1

N2 - Background: Pre-planned futility analyses are commonly used in oncology studies. The LUME-Lung 2 study (NCT00806819; 1199.14) was stopped early based on a pre-planned, non-binding futility analysis of investigator-assessed progression-free survival (PFS), although subsequent analysis showed that the primary endpoint of improvement in centrally reviewed PFS was met. Retrospective analyses were conducted to understand the discrepancy between interim futility and final analyses.Materials and methods: LUME-Lung 2 investigated nintedanib in combination with pemetrexed versus placebo‒pemetrexed for the treatment of patients with advanced or recurrent non-squamous non-small cell lung cancer who had relapsed or failed one prior line of chemotherapy. Pre-planned futility analysis was carried out by the Data Monitoring Committee (DMC) after 50% of the events for the primary PFS analysis (713 events) had occurred; the threshold for futility was a conditional power of < 20%. Conditional/predictive powers and hazard ratios were calculated retrospectively after varying percentages of events had occurred for both investigator- and centrally reviewed PFS.Results: At the time of the pre-planned futility analysis, the conditional power was 10.3% and the predictive power was 18.5%; no safety issues were identified. Retrospective analysis showed that the conditional and predictive powers fluctuated considerably over time for both investigator- and centrally reviewed PFS and that the power only dropped by a notable amount, and below the futility threshold, at the time of the futility analysis.Conclusions: Retrospective investigations suggest that, had the DMC analysis been carried out at another time point, or had centrally reviewed PFS data been used, the futility outcome may have been different and the trial may have been continued. The design of futility analyses requires careful consideration and confirming negative futility outcomes by second analysis may be appropriate.Trial number: NCT00806819.

AB - Background: Pre-planned futility analyses are commonly used in oncology studies. The LUME-Lung 2 study (NCT00806819; 1199.14) was stopped early based on a pre-planned, non-binding futility analysis of investigator-assessed progression-free survival (PFS), although subsequent analysis showed that the primary endpoint of improvement in centrally reviewed PFS was met. Retrospective analyses were conducted to understand the discrepancy between interim futility and final analyses.Materials and methods: LUME-Lung 2 investigated nintedanib in combination with pemetrexed versus placebo‒pemetrexed for the treatment of patients with advanced or recurrent non-squamous non-small cell lung cancer who had relapsed or failed one prior line of chemotherapy. Pre-planned futility analysis was carried out by the Data Monitoring Committee (DMC) after 50% of the events for the primary PFS analysis (713 events) had occurred; the threshold for futility was a conditional power of < 20%. Conditional/predictive powers and hazard ratios were calculated retrospectively after varying percentages of events had occurred for both investigator- and centrally reviewed PFS.Results: At the time of the pre-planned futility analysis, the conditional power was 10.3% and the predictive power was 18.5%; no safety issues were identified. Retrospective analysis showed that the conditional and predictive powers fluctuated considerably over time for both investigator- and centrally reviewed PFS and that the power only dropped by a notable amount, and below the futility threshold, at the time of the futility analysis.Conclusions: Retrospective investigations suggest that, had the DMC analysis been carried out at another time point, or had centrally reviewed PFS data been used, the futility outcome may have been different and the trial may have been continued. The design of futility analyses requires careful consideration and confirming negative futility outcomes by second analysis may be appropriate.Trial number: NCT00806819.

KW - futility analysis

KW - methodology

KW - modelling, biostatistics

KW - non-small cell lung cancer

UR - http://www.scopus.com/inward/record.url?scp=85041265004&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85041265004&partnerID=8YFLogxK

U2 - 10.1093/annonc/mdx042

DO - 10.1093/annonc/mdx042

M3 - Article

C2 - 28184431

AN - SCOPUS:85041265004

VL - 28

SP - 1419

EP - 1426

JO - Annals of Oncology

JF - Annals of Oncology

SN - 0923-7534

IS - 7

ER -