Survival analysis: Methods for analyzing data with censored observations

Tomasz Burzykowski

DOI: https://doi.org/10.1053/j.sodo.2024.01.008

IF: 1.34

2024-02-03

Seminars in Orthodontics

Abstract:This article provides a review of the basic statistical methods for the analysis of data that include censored observations.

dentistry, oral surgery & medicine

What problem does this paper attempt to address?

SURVIVAL ANALYSIS IN PUBLIC HEALTH RESEARCH

Elisa T. Lee,Oscar T. Go

DOI: https://doi.org/10.1146/annurev.publhealth.18.1.105

IF: 21.87

1997-05-01

Annual Review of Public Health

Abstract:▪ Abstract This paper reviews the common statistical techniques employed to analyze survival data in public health research. Due to the presence of censoring, the data are not amenable to the usual method of analysis. The improvement in statistical computing and wide accessibility of personal computers led to the rapid development and popularity of nonparametric over parametric procedures. The former required less stringent conditions. But, if the assumptions for parametric methods hold, the resulting estimates have smaller standard errors and are easier to interpret. Nonparametric techniques include the Kaplan-Meier method for estimating the survival function and the Cox proportional hazards model to identify risk factors and to obtain adjusted risk ratios. In cases where the assumption of proportional hazards is not tenable, the data can be stratified and a model fitted with different baseline functions in each stratum. Parametric modeling such as the accelerated failure time model also may be used. Hazard functions for the exponential, Weibull, gamma, Gompertz, lognormal, and log-logistic distributions are described. Examples from published literature are given to illustrate the various methods. The paper is intended for public health professionals who are interested in survival data analysis.

public, environmental & occupational health
Machine Learning for Survival Analysis: A Survey

Ping Wang,Yan Li,Chandan K. Reddy

DOI: https://doi.org/10.48550/arXiv.1708.04649

IF: 5.414

2017-08-15

Machine Learning

Abstract:Accurately predicting the time of occurrence of an event of interest is a critical problem in longitudinal data analysis. One of the main challenges in this context is the presence of instances whose event outcomes become unobservable after a certain time point or when some instances do not experience any event during the monitoring period. Such a phenomenon is called censoring which can be effectively handled using survival analysis techniques. Traditionally, statistical approaches have been widely developed in the literature to overcome this censoring issue. In addition, many machine learning algorithms are adapted to effectively handle survival data and tackle other challenging problems that arise in real-world data. In this survey, we provide a comprehensive and structured review of the representative statistical methods along with the machine learning techniques used in survival analysis and provide a detailed taxonomy of the existing methods. We also discuss several topics that are closely related to survival analysis and illustrate several successful applications in various real-world application domains. We hope that this paper will provide a more thorough understanding of the recent advances in survival analysis and offer some guidelines on applying these approaches to solve new problems that arise in applications with censored data.
Statistical Description for Survival Data.

Zhongheng Zhang

DOI: https://doi.org/10.21037/atm.2016.07.17

IF: 3.616

2016-01-01

Annals of Translational Medicine

Abstract:Statistical description is always the first step in data analysis. It gives investigator a general impression of the data at hand. Traditionally, data are described as central tendency and deviation. However, this framework does not fit to the survival data (also termed time-to-event data). Such data type contains two components. One is the survival time and the other is the status. Researchers are usually interested in the probability of event at a given survival time point. Hazard function, cumulative hazard function and survival function are commonly used to describe survival data. Survival function can be estimated using Kaplan-Meier estimator, which is also the default method in most statistical packages. Alternatively, Nelson-Aalen estimator is available to estimate survival function. Survival functions of subgroups can be compared using log-rank test. Furthermore, the article also introduces how to describe time-to-event data with parametric modeling.
Survival analysis: Part I — analysis of time-to-event

Junyong In,Dong Kyu Lee

DOI: https://doi.org/10.4097/kja.d.18.00067

IF: 5.167

2018-06-01

Korean Journal of Anesthesiology

Abstract:Length of time is a variable often encountered during data analysis. Survival analysis provides simple, intuitive results concerning time-to-event for events of interest, which are not confined to death. This review introduces methods of analyzing time-to-event. The Kaplan-Meier survival analysis, log-rank test, and Cox proportional hazards regression modeling method are described with examples of hypothetical data.
Review of statistical methods for survival analysis using genomic data

Seungyeoun Lee,Heeju Lim

DOI: https://doi.org/10.5808/gi.2019.17.4.e41

2019-12-31

Abstract:Survival analysis mainly deals with the time to event, including death, onset of disease, and bankruptcy. The common characteristic of survival analysis is that it contains "censored" data, in which the time to event cannot be completely observed, but instead represents the lower bound of the time to event. Only the occurrence of either time to event or censoring time is observed. Many traditional statistical methods have been effectively used for analyzing survival data with censored observations. However, with the development of high-throughput technologies for producing "omics" data, more advanced statistical methods, such as regularization, should be required to construct the predictive survival model with high-dimensional genomic data. Furthermore, machine learning approaches have been adapted for survival analysis, to fit nonlinear and complex interaction effects between predictors, and achieve more accurate prediction of individual survival probability. Presently, since most clinicians and medical researchers can easily assess statistical programs for analyzing survival data, a review article is helpful for understanding statistical methods used in survival analysis. We review traditional survival methods and regularization methods, with various penalty functions, for the analysis of high-dimensional genomics, and describe machine learning techniques that have been adapted to survival analysis.
Survival Analysis and Interpretation of Time-to-Event Data: The Tortoise and the Hare

Patrick Schober,Thomas R. Vetter,Vetter, Thomas R.,Schober, Patrick

DOI: https://doi.org/10.1213/ANE.0000000000003653

2019-10-03

Abstract:Survival analysis, or more generally, time-to-event analysis, refers to a set of methods for analyzing the length of time until the occurrence of a well-defined end point of interest. A unique feature of survival data is that typically not all patients experience the event (eg, death) by the end of the observation period, so the actual survival times for some patients are unknown. This phenomenon, referred to as censoring, must be accounted for in the analysis to allow for valid inferences. Moreover, survival times are usually skewed, limiting the usefulness of analysis methods that assume a normal data distribution. As part of the ongoing series in Anesthesia & Analgesia, this tutorial reviews statistical methods for the appropriate analysis of time-to-event data, including nonparametric and semiparametric methods—specifically the Kaplan-Meier estimator, log-rank test, and Cox proportional hazards model. These methods are by far the most commonly used techniques for such data in medical literature. Illustrative examples from studies published in Anesthesia & Analgesia demonstrate how these techniques are used in practice. Full parametric models and models to deal with special circumstances, such as recurrent events models, competing risks models, and frailty models, are briefly discussed.

anesthesiology
Sampling with censored data: a practical guide

Pedro L. Ramos,Daniel C. F. Guzman,Alex L. Mota,Daniel A. Saavedra,Francisco A. Rodrigues,Francisco Louzada

DOI: https://doi.org/10.1080/00949655.2024.2409379

IF: 1.225

2024-10-11

Journal of Statistical Computation and Simulation

Abstract:In this review, we present a simple guide for researchers to obtain pseudo-random samples with censored data. We focus our attention on the most common types of censored data, such as type I, type II, and random censoring. We discussed the necessary steps to sample pseudo-random values from long-term survival models where an additional cure fraction is informed. For illustrative purposes, these techniques are applied in the Weibull distribution. The algorithms and codes in R are presented, enabling the reproducibility of our study. Finally, we developed an R package that encapsulates these methodologies, providing researchers with practical tools for implementation.

statistics & probability,computer science, interdisciplinary applications
Survival Analysis Using SAS: A Practical Guide, Second Edition

P. Allison

1995-11-13

Abstract:Easy to read and comprehensive, Survival Analysis Using SAS: A Practical Guide, Second Edition, by Paul D. Allison, is an accessible, data-based introduction to methods of survival analysis. Researchers who want to analyze survival data with SAS will find just what they need with this fully updated new edition that incorporates the many enhancements in SAS procedures for survival analysis in SAS 9. Although the book assumes only a minimal knowledge of SAS, more experienced users will learn new techniques of data input and manipulation. Numerous examples of SAS code and output make this an eminently practical book, ensuring that even the uninitiated become sophisticated users of survival analysis. The main topics presented include censoring, survival curves, Kaplan-Meier estimation, accelerated failure time models, Cox regression models, and discrete-time analysis. Also included are topics not usually covered in survival analysis books, such as time-dependent covariates, competing risks, and repeated events. Survival Analysis Using SAS: A Practical Guide, Second Edition, has been thoroughly updated for SAS 9, and all figures are presented using ODS graphics. This new edition also documents major enhancements to the STRATA statement in the LIFETEST procedure; includes a section on the PROBPLOT command, which offers graphical methods to evaluate the fit of each parametric regression model; introduces the new BAYES statement for both parametric and Cox models, which allows the user to do a Bayesian analysis using MCMC methods; demonstrates the use of the counting process syntax as an alternative method for handling time-dependent covariates; contains a section on cumulative incidence functions; and describes the use of the new GLIMMIX procedure to estimate random-effects models for discrete-time data.

Computer Science
Statistical Methods for Conditional Survival Analysis

Sin-Ho Jung,Ho Yun Lee,Shein-Chung Chow

DOI: https://doi.org/10.1080/10543406.2017.1405012

2017-11-29

Journal of Biopharmaceutical Statistics

Abstract:We investigate the survival distribution of the patients who have survived over a certain time period. This is called a conditional survival distribution. In this paper, we show that one-sample estimation, two-sample comparison and regression analysis of conditional survival distributions can be conducted using the regular methods for unconditional survival distributions that are provided by the standard statistical software, such as SAS and SPSS. We conduct extensive simulations to evaluate the finite sample property of these conditional survival analysis methods. We illustrate these methods with real clinical data.

pharmacology & pharmacy,statistics & probability
Statistical Methods for Survival Data Analysis

Elisa T. Lee

DOI: https://doi.org/10.2307/2533355

IF: 5.883

1994-12-01

IEEE Transactions on Reliability

Abstract:Praise for the Third Edition. . . an easy-to read introduction to survival analysis which covers the major concepts and techniques of the subject. Statistics in Medical ResearchUpdated and expanded to reflect the latest developments, Statistical Methods for Survival Data Analysis, Fourth Edition continues to deliver a comprehensive introduction to the most commonly-used methods for analyzing survival data. Authored by a uniquely well-qualified author team, the Fourth Edition is a critically acclaimed guide to statistical methods with applications in clinical trials, epidemiology, areas of business, and the social sciences. The book features many real-world examples to illustrate applications within these various fields, although special consideration is given to the study of survival data in biomedical sciences.Emphasizing the latest research and providing the most up-to-date information regarding software applications in the field, Statistical Methods for Survival Data Analysis, Fourth Edition also includes:Marginal and random effect models for analyzing correlated censored or uncensored dataMultiple types of two-sample and K-sample comparison analysisUpdated treatment of parametric methods for regression model fitting with a new focus on accelerated failure time modelsExpanded coverage of the Cox proportional hazards modelExercises at the end of each chapter to deepen knowledge of the presented materialStatistical Methods for Survival Data Analysis is an ideal text for upper-undergraduate and graduate-level courses on survival data analysis. The book is also an excellent resource for biomedical investigators, statisticians, and epidemiologists, as well as researchers in every field in which the analysis of survival data plays a role.

Mathematics,Medicine,Computer Science,Business
Survival Trees for Interval-Censored Survival data

Wei Fu,Jeffrey S. Simonoff

DOI: https://doi.org/10.48550/arXiv.1702.07763

2017-07-20

Abstract:Interval-censored data, in which the event time is only known to lie in some time interval, arise commonly in practice; for example, in a medical study in which patients visit clinics or hospitals at pre-scheduled times, and the events of interest occur between visits. Such data are appropriately analyzed using methods that account for this uncertainty in event time measurement. In this paper we propose a survival tree method for interval-censored data based on the conditional inference framework. Using Monte Carlo simulations we find that the tree is effective in uncovering underlying tree structure, performs similarly to an interval-censored Cox proportional hazards model fit when the true relationship is linear, and performs at least as well as (and in the presence of right-censoring outperforms) the Cox model when the true relationship is not linear. Further, the interval-censored tree outperforms survival trees based on imputing the event time as an endpoint or the midpoint of the censoring interval. We illustrate the application of the method on tooth emergence data.

Methodology
Advanced considerations in survival analysis

Manuel Carnero-Alcázar,Lourdes Montero-Cruces,Javier Cobiella-Carnicer,Daniel Pérez-Camargo,Luis Maroto Castellanos

DOI: https://doi.org/10.1093/ejcts/ezae020

IF: 4.534

2024-01-19

European Journal of Cardio-Thoracic Surgery

Abstract:Abstract Investigation of survival during the follow-up period is common in cardiovascular research and has intrinsic issues that require precise knowledge, such as survival or censoring. Besides, as the follow-up period lengthens and events other than mortality are studied, the analysis becomes more complex, so Kaplan–Meier analyses or Cox models are not always sufficient. In this primer, we provide the reader with detailed information on the interpretation of the most common survival analyses and delve into methods to analyse competing risks or alternatives to the conventional methods when the proportional hazards assumption is not met.

surgery,cardiac & cardiovascular systems,respiratory system
Copula-based Semiparametric Nonnormal Transformed Linear Model for Survival Data with Dependent Censoring

Huazhen Yu,Lixin Zhang

2024-01-01

Abstract:Although the independent censoring assumption is commonly used in survival analysis, it can be violated when the censoring time is related to the survival time, which often happens in many practical applications. To address this issue, we propose a flexible semiparametric method for dependent censored data. Our approach involves fitting the survival time and the censoring time with a joint transformed linear model, where the transformed function is unspecified. This allows for a very general class of models that can account for possible covariate effects, while also accommodating administrative censoring. We assume that the transformed variables have a bivariate nonnormal distribution based on parametric copulas and parametric marginals, which further enhances the flexibility of our method. We demonstrate the identifiability of the proposed model and establish the consistency and asymptotic normality of the model parameters under appropriate regularity conditions and assumptions. Furthermore, we evaluate the performance of our method through extensive simulation studies, and provide a real data example for illustration.
An Overview of Introductory and Advanced Survival Analysis Methods in Clinical Applications: Where Have we Come so far?

Georgios Beis,Aggelos Iliopoulos,Ioannis Papasotiriou,GEORGIOS BEIS,AGGELOS ILIOPOULOS,IOANNIS PAPASOTIRIOU

DOI: https://doi.org/10.21873/anticanres.16835

2024-02-01

Anticancer Research

Abstract:The time-to-event relationship for survival modeling is considered when designing a study in clinical trials. However, because time-to-event data are mostly not normally distributed, survival analysis uses non-parametric data processing and analysis methods, mainly Kaplan-Meier (KM) estimation models and Cox proportional hazards (CPH) regression models. At the same time, the log-rank test can be applied to compare curves from different groups. However, resorting to conventional survival analysis when fundamental assumptions, such as the Cox PH assumption, are not met can seriously affect the results, rendering them flawed. Consequently, it is necessary to examine and report more sophisticated statistical methods related to the processing of survival data, but at the same time, able to adequately respond to the contemporary real problems of clinical applications. On the other hand, the frequent misinterpretation of survival analysis methodology, combined with the fact that it is a complex statistical tool for clinicians, necessitates a better understanding of the basic principles underlying this analysis to effectively interpret medical studies in making treatment decisions. In this review, we first consider the basic models and mechanisms behind survival analysis. Then, due to common errors arising from the inappropriate application of conventional models, we revise more demanding statistical extensions of survival models related to data manipulation to avoid wrong results. By providing a structured review of the most representative statistical methods and tests covering contemporary survival analysis, we hope this review will assist in solving problems that arise in clinical applications.

oncology
Learning to rank for censored survival data

Margaux Luck,Tristan Sylvain,Joseph Paul Cohen,Heloise Cardinal,Andrea Lodi,Yoshua Bengio

DOI: https://doi.org/10.48550/arXiv.1806.01984

2018-06-09

Abstract:Survival analysis is a type of semi-supervised ranking task where the target output (the survival time) is often right-censored. Utilizing this information is a challenge because it is not obvious how to correctly incorporate these censored examples into a model. We study how three categories of loss functions, namely partial likelihood methods, rank methods, and our classification method based on a Wasserstein metric (WM) and the non-parametric Kaplan Meier estimate of the probability density to impute the labels of censored examples, can take advantage of this information. The proposed method allows us to have a model that predict the probability distribution of an event. If a clinician had access to the detailed probability of an event over time this would help in treatment planning. For example, determining if the risk of kidney graft rejection is constant or peaked after some time. Also, we demonstrate that this approach directly optimizes the expected C-index which is the most common evaluation metric for ranking survival models.

Machine Learning,Artificial Intelligence
A discrete-time survival model to handle interval-censored covariates

Avi Kenny,Stephen Olivier,James P. Hughes,Mark J. Siedner

DOI: https://doi.org/10.48550/arXiv.2408.07738

2024-08-15

Abstract:Methods are lacking to handle the problem of survival analysis in the presence of an interval-censored covariate, specifically the case in which the conditional hazard of the primary event of interest depends on the occurrence of a secondary event, the observation time of which is subject to interval censoring. We propose and study a flexible class of discrete-time parametric survival models that handle the censoring problem through joint modeling of the interval-censored secondary event, the outcome, and the censoring mechanism. We apply this model to the research question that motivated the methodology, estimating the effect of HIV status on all-cause mortality in a prospective cohort study in South Africa.

Methodology
Modeling time-to-event (survival) data using classification tree analysis

Ariel Linden,Paul R Yarnold

DOI: https://doi.org/10.1111/jep.12779

Abstract:Rationale, aims, and objectives: Time to the occurrence of an event is often studied in health research. Survival analysis differs from other designs in that follow-up times for individuals who do not experience the event by the end of the study (called censored) are accounted for in the analysis. Cox regression is the standard method for analysing censored data, but the assumptions required of these models are easily violated. In this paper, we introduce classification tree analysis (CTA) as a flexible alternative for modelling censored data. Classification tree analysis is a "decision-tree"-like classification model that provides parsimonious, transparent (ie, easy to visually display and interpret) decision rules that maximize predictive accuracy, derives exact P values via permutation tests, and evaluates model cross-generalizability. Method: Using empirical data, we identify all statistically valid, reproducible, longitudinally consistent, and cross-generalizable CTA survival models and then compare their predictive accuracy to estimates derived via Cox regression and an unadjusted naïve model. Model performance is assessed using integrated Brier scores and a comparison between estimated survival curves. Results: The Cox regression model best predicts average incidence of the outcome over time, whereas CTA survival models best predict either relatively high, or low, incidence of the outcome over time. Conclusions: Classification tree analysis survival models offer many advantages over Cox regression, such as explicit maximization of predictive accuracy, parsimony, statistical robustness, and transparency. Therefore, researchers interested in accurate prognoses and clear decision rules should consider developing models using the CTA-survival framework.
Statistical Issues on Analysis of Censored Data Due to Detection Limit

Hua He,Xuenan Mi,Jerry Cornell,Wan Tang,Tanika Kelly,Hui Shen,Hongwen Deng,Yan Du

DOI: https://doi.org/10.5539/ijsp.v9n4p49

2020-06-14

International Journal of Statistics and Probability

Abstract:Measures of substance concentration in urine, serum or other biological matrices often have an assay limit of detection. When concentration levels fall below the limit, the exact measures cannot be obtained, and thus are left censored. Common practice for addressing the censoring issue is to delete or 'fill-in' the censored observations in data analysis, which often results in biased or non-efficient estimates. Assuming the concentration or transformed concentration follows a normal distribution, a Tobit regression model can be applied. When the study population is heterogeneous, for example due to the existence of a latent group of subjects who lack the substance, the problem becomes more challenging. In this paper, we conduct intensive simulation studies to investigate the statistical issues in analyzing censored data and compare different methods in which the data are treated either as a dependent variable or an independent variable. We also analyze triclosan data in the NHANES study and metabolites data in the Bogalusa Heart Study to illustrate the issues. Some guidelines for analyzing such censored data are provided.

Survival analysis: Methods for analyzing data with censored observations

SURVIVAL ANALYSIS IN PUBLIC HEALTH RESEARCH

Machine Learning for Survival Analysis: A Survey

Statistical Description for Survival Data.

Survival analysis: Part I — analysis of time-to-event

Review of statistical methods for survival analysis using genomic data

Survival Analysis and Interpretation of Time-to-Event Data: The Tortoise and the Hare

Sampling with censored data: a practical guide

Survival Analysis Using SAS: A Practical Guide, Second Edition

Statistical Methods for Conditional Survival Analysis

Statistical Methods for Survival Data Analysis

Survival Trees for Interval-Censored Survival data

Advanced considerations in survival analysis

Copula-based Semiparametric Nonnormal Transformed Linear Model for Survival Data with Dependent Censoring

An Overview of Introductory and Advanced Survival Analysis Methods in Clinical Applications: Where Have we Come so far?

Learning to rank for censored survival data

A discrete-time survival model to handle interval-censored covariates

Modeling time-to-event (survival) data using classification tree analysis

Statistical Issues on Analysis of Censored Data Due to Detection Limit