Control of false positives in randomized phase III clinical trials

Changyu Shen, Ziyue Liu, Huiping Xu, Hai Liu, Cynthia Yue

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Randomized Phase III clinical trials serve as the gold-standard for the evaluation of the efficacy of a medical intervention. Although research and development in earlier stages together with rigorous statistical examination assure a small probability of false positive for a given trial, it is unclear how many false positives were generated from the large number of randomized Phase III trials from the biopharmaceutical industry in the United States. The proportion of comparisons in Phase III trials where the medical intervention has null or negative efficacy, or proportion of null or negative (PNN), is at the central position for the estimation and control of the number of false positives. We seek to estimate PNN using a new Bayesian deconvolution method. Using data from clinicaltrials.gov and other data sources, we identified 1393 trials completed in 2008–2012 that meet our study entry criteria, which are dominated by trials on drugs for treatment purpose. Among the 1221 trials with results available on the selected comparisons, 789 (64.6%) show statistically significant superiority of the intervention, with 561 (45.9%) having a two-sided p-value less than 0.001. The PNN is estimated to be no more than 7–9%. Based on the PNN, we estimated that 18–22% of the trials have at least one comparison with null or negative efficacy, leading to an expectation of no more than 6–8 trials with at least one false positive comparison over a 5-year period.

Original languageEnglish (US)
Pages (from-to)1-13
Number of pages13
JournalJournal of Biopharmaceutical Statistics
DOIs
StateAccepted/In press - Sep 22 2016

Fingerprint

Phase III Clinical Trials
Bayes Theorem
Information Storage and Retrieval
False Positive
Gold
Clinical Trials
Industry
Randomized Controlled Trials
Null
Proportion
Research
Pharmaceutical Preparations
Efficacy
Deconvolution
p-Value
Research and Development
Drugs
Evaluation

Keywords

  • Bayesian deconvolution
  • false positive control
  • phase III
  • randomized trials

ASJC Scopus subject areas

  • Statistics and Probability
  • Pharmacology
  • Pharmacology (medical)

Cite this

Control of false positives in randomized phase III clinical trials. / Shen, Changyu; Liu, Ziyue; Xu, Huiping; Liu, Hai; Yue, Cynthia.

In: Journal of Biopharmaceutical Statistics, 22.09.2016, p. 1-13.

Research output: Contribution to journalArticle

@article{f8f4b140b7444c49a77796c8ff9c2477,
title = "Control of false positives in randomized phase III clinical trials",
abstract = "Randomized Phase III clinical trials serve as the gold-standard for the evaluation of the efficacy of a medical intervention. Although research and development in earlier stages together with rigorous statistical examination assure a small probability of false positive for a given trial, it is unclear how many false positives were generated from the large number of randomized Phase III trials from the biopharmaceutical industry in the United States. The proportion of comparisons in Phase III trials where the medical intervention has null or negative efficacy, or proportion of null or negative (PNN), is at the central position for the estimation and control of the number of false positives. We seek to estimate PNN using a new Bayesian deconvolution method. Using data from clinicaltrials.gov and other data sources, we identified 1393 trials completed in 2008–2012 that meet our study entry criteria, which are dominated by trials on drugs for treatment purpose. Among the 1221 trials with results available on the selected comparisons, 789 (64.6{\%}) show statistically significant superiority of the intervention, with 561 (45.9{\%}) having a two-sided p-value less than 0.001. The PNN is estimated to be no more than 7–9{\%}. Based on the PNN, we estimated that 18–22{\%} of the trials have at least one comparison with null or negative efficacy, leading to an expectation of no more than 6–8 trials with at least one false positive comparison over a 5-year period.",
keywords = "Bayesian deconvolution, false positive control, phase III, randomized trials",
author = "Changyu Shen and Ziyue Liu and Huiping Xu and Hai Liu and Cynthia Yue",
year = "2016",
month = "9",
day = "22",
doi = "10.1080/10543406.2016.1222536",
language = "English (US)",
pages = "1--13",
journal = "Journal of Biopharmaceutical Statistics",
issn = "1054-3406",
publisher = "Taylor and Francis Ltd.",

}

TY - JOUR

T1 - Control of false positives in randomized phase III clinical trials

AU - Shen, Changyu

AU - Liu, Ziyue

AU - Xu, Huiping

AU - Liu, Hai

AU - Yue, Cynthia

PY - 2016/9/22

Y1 - 2016/9/22

N2 - Randomized Phase III clinical trials serve as the gold-standard for the evaluation of the efficacy of a medical intervention. Although research and development in earlier stages together with rigorous statistical examination assure a small probability of false positive for a given trial, it is unclear how many false positives were generated from the large number of randomized Phase III trials from the biopharmaceutical industry in the United States. The proportion of comparisons in Phase III trials where the medical intervention has null or negative efficacy, or proportion of null or negative (PNN), is at the central position for the estimation and control of the number of false positives. We seek to estimate PNN using a new Bayesian deconvolution method. Using data from clinicaltrials.gov and other data sources, we identified 1393 trials completed in 2008–2012 that meet our study entry criteria, which are dominated by trials on drugs for treatment purpose. Among the 1221 trials with results available on the selected comparisons, 789 (64.6%) show statistically significant superiority of the intervention, with 561 (45.9%) having a two-sided p-value less than 0.001. The PNN is estimated to be no more than 7–9%. Based on the PNN, we estimated that 18–22% of the trials have at least one comparison with null or negative efficacy, leading to an expectation of no more than 6–8 trials with at least one false positive comparison over a 5-year period.

AB - Randomized Phase III clinical trials serve as the gold-standard for the evaluation of the efficacy of a medical intervention. Although research and development in earlier stages together with rigorous statistical examination assure a small probability of false positive for a given trial, it is unclear how many false positives were generated from the large number of randomized Phase III trials from the biopharmaceutical industry in the United States. The proportion of comparisons in Phase III trials where the medical intervention has null or negative efficacy, or proportion of null or negative (PNN), is at the central position for the estimation and control of the number of false positives. We seek to estimate PNN using a new Bayesian deconvolution method. Using data from clinicaltrials.gov and other data sources, we identified 1393 trials completed in 2008–2012 that meet our study entry criteria, which are dominated by trials on drugs for treatment purpose. Among the 1221 trials with results available on the selected comparisons, 789 (64.6%) show statistically significant superiority of the intervention, with 561 (45.9%) having a two-sided p-value less than 0.001. The PNN is estimated to be no more than 7–9%. Based on the PNN, we estimated that 18–22% of the trials have at least one comparison with null or negative efficacy, leading to an expectation of no more than 6–8 trials with at least one false positive comparison over a 5-year period.

KW - Bayesian deconvolution

KW - false positive control

KW - phase III

KW - randomized trials

UR - http://www.scopus.com/inward/record.url?scp=84988311572&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84988311572&partnerID=8YFLogxK

U2 - 10.1080/10543406.2016.1222536

DO - 10.1080/10543406.2016.1222536

M3 - Article

C2 - 27653151

AN - SCOPUS:84988311572

SP - 1

EP - 13

JO - Journal of Biopharmaceutical Statistics

JF - Journal of Biopharmaceutical Statistics

SN - 1054-3406

ER -