Maximum likelihood estimations and EM algorithms with length-biased data

Jing Qin, Jing Ning, Hao Liu, Yu Shen

Research output: Contribution to journalArticle

32 Citations (Scopus)

Abstract

Length-biased sampling has been well recognized in economics, industrial reliability, etiology applications, and epidemiological, genetic, and cancer screening studies. Length-biased right-censored data have a unique data structure different from traditional survival data. The nonparametric and semiparametric estimation and inference methods for traditional survival data are not directly applicable for lengthbiased right-censored data. We propose new expectation-maximization algorithms for estimations based on full likelihoods involving infinite-dimensional parameters under three settings for length-biased data: estimating nonparametric distribution function, estimating nonparametric hazard function under an increasing failure rate constraint, and jointly estimating baseline hazards function and the covariate coefficients under the Cox proportional hazards model. Extensive empirical simulation studies show that the maximum likelihood estimators perform well with moderate sample sizes and lead to more efficient estimators compared to the estimating equation approaches. The proposed estimates are also more robust to various right-censoring mechanisms. We prove the strong consistency properties of the estimators, and establish the asymptotic normality of the semiparametric maximum likelihood estimators under the Cox model using modern empirical processes theory. We apply the proposed methods to a prevalent cohort medical study. Supplemental materials are available online.

Original languageEnglish (US)
Pages (from-to)1434-1449
Number of pages16
JournalJournal of the American Statistical Association
Volume106
Issue number496
DOIs
StatePublished - Dec 1 2011
Externally publishedYes

Fingerprint

EM Algorithm
Estimation Algorithms
Maximum Likelihood Estimation
Biased
Right-censored Data
Hazard Function
Survival Data
Maximum Likelihood Estimator
Semiparametric Inference
Biased Sampling
Increasing Failure Rate
Right Censoring
Semiparametric Estimation
Cox Proportional Hazards Model
Cox Model
Efficient Estimator
Strong Consistency
Estimating Equation
Empirical Process
Expectation-maximization Algorithm

Keywords

  • Cox regression model
  • Increasing failure rate
  • Nonparametric likelihood
  • Profile likelihood
  • Right-censored data

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Maximum likelihood estimations and EM algorithms with length-biased data. / Qin, Jing; Ning, Jing; Liu, Hao; Shen, Yu.

In: Journal of the American Statistical Association, Vol. 106, No. 496, 01.12.2011, p. 1434-1449.

Research output: Contribution to journalArticle

@article{3d9df1aa39cb4cd780031984405ca448,
title = "Maximum likelihood estimations and EM algorithms with length-biased data",
abstract = "Length-biased sampling has been well recognized in economics, industrial reliability, etiology applications, and epidemiological, genetic, and cancer screening studies. Length-biased right-censored data have a unique data structure different from traditional survival data. The nonparametric and semiparametric estimation and inference methods for traditional survival data are not directly applicable for lengthbiased right-censored data. We propose new expectation-maximization algorithms for estimations based on full likelihoods involving infinite-dimensional parameters under three settings for length-biased data: estimating nonparametric distribution function, estimating nonparametric hazard function under an increasing failure rate constraint, and jointly estimating baseline hazards function and the covariate coefficients under the Cox proportional hazards model. Extensive empirical simulation studies show that the maximum likelihood estimators perform well with moderate sample sizes and lead to more efficient estimators compared to the estimating equation approaches. The proposed estimates are also more robust to various right-censoring mechanisms. We prove the strong consistency properties of the estimators, and establish the asymptotic normality of the semiparametric maximum likelihood estimators under the Cox model using modern empirical processes theory. We apply the proposed methods to a prevalent cohort medical study. Supplemental materials are available online.",
keywords = "Cox regression model, Increasing failure rate, Nonparametric likelihood, Profile likelihood, Right-censored data",
author = "Jing Qin and Jing Ning and Hao Liu and Yu Shen",
year = "2011",
month = "12",
day = "1",
doi = "10.1198/jasa.2011.tm10156",
language = "English (US)",
volume = "106",
pages = "1434--1449",
journal = "Journal of the American Statistical Association",
issn = "0162-1459",
publisher = "Taylor and Francis Ltd.",
number = "496",

}

TY - JOUR

T1 - Maximum likelihood estimations and EM algorithms with length-biased data

AU - Qin, Jing

AU - Ning, Jing

AU - Liu, Hao

AU - Shen, Yu

PY - 2011/12/1

Y1 - 2011/12/1

N2 - Length-biased sampling has been well recognized in economics, industrial reliability, etiology applications, and epidemiological, genetic, and cancer screening studies. Length-biased right-censored data have a unique data structure different from traditional survival data. The nonparametric and semiparametric estimation and inference methods for traditional survival data are not directly applicable for lengthbiased right-censored data. We propose new expectation-maximization algorithms for estimations based on full likelihoods involving infinite-dimensional parameters under three settings for length-biased data: estimating nonparametric distribution function, estimating nonparametric hazard function under an increasing failure rate constraint, and jointly estimating baseline hazards function and the covariate coefficients under the Cox proportional hazards model. Extensive empirical simulation studies show that the maximum likelihood estimators perform well with moderate sample sizes and lead to more efficient estimators compared to the estimating equation approaches. The proposed estimates are also more robust to various right-censoring mechanisms. We prove the strong consistency properties of the estimators, and establish the asymptotic normality of the semiparametric maximum likelihood estimators under the Cox model using modern empirical processes theory. We apply the proposed methods to a prevalent cohort medical study. Supplemental materials are available online.

AB - Length-biased sampling has been well recognized in economics, industrial reliability, etiology applications, and epidemiological, genetic, and cancer screening studies. Length-biased right-censored data have a unique data structure different from traditional survival data. The nonparametric and semiparametric estimation and inference methods for traditional survival data are not directly applicable for lengthbiased right-censored data. We propose new expectation-maximization algorithms for estimations based on full likelihoods involving infinite-dimensional parameters under three settings for length-biased data: estimating nonparametric distribution function, estimating nonparametric hazard function under an increasing failure rate constraint, and jointly estimating baseline hazards function and the covariate coefficients under the Cox proportional hazards model. Extensive empirical simulation studies show that the maximum likelihood estimators perform well with moderate sample sizes and lead to more efficient estimators compared to the estimating equation approaches. The proposed estimates are also more robust to various right-censoring mechanisms. We prove the strong consistency properties of the estimators, and establish the asymptotic normality of the semiparametric maximum likelihood estimators under the Cox model using modern empirical processes theory. We apply the proposed methods to a prevalent cohort medical study. Supplemental materials are available online.

KW - Cox regression model

KW - Increasing failure rate

KW - Nonparametric likelihood

KW - Profile likelihood

KW - Right-censored data

UR - http://www.scopus.com/inward/record.url?scp=84862953577&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84862953577&partnerID=8YFLogxK

U2 - 10.1198/jasa.2011.tm10156

DO - 10.1198/jasa.2011.tm10156

M3 - Article

VL - 106

SP - 1434

EP - 1449

JO - Journal of the American Statistical Association

JF - Journal of the American Statistical Association

SN - 0162-1459

IS - 496

ER -