Abstract:Background: More than 95% of recent cancer randomized controlled trials used the log-rank test to detect a treatment difference making it the predominant tool for comparing two survival functions. As with other tests, the log-rank test has both advantages and disadvantages. One advantage is that it offers the highest power against proportional hazards differences, which may be a major reason why alternative methods have rarely been employed in practice. The performance of statistical tests has traditionally been investigated both theoretically and numerically for several patterns of difference between two survival functions. However, to the best of our knowledge, there has been no attempt to compare the performance of various statistical tests using empirical data from past oncology randomized controlled trials. So, it is unknown whether the log-rank test offers a meaningful power advantage over alternative testing methods in contemporary cancer randomized controlled trials. Focusing on recently reported phase III cancer randomized controlled trials, we assessed whether the log-rank test gave meaningfully greater power when compared with five alternative testing methods: generalized Wilcoxon, test based on maximum of test statistics from multiple weighted log-rank tests, difference in t-year event rate, and difference in restricted mean survival time with fixed and adaptive τ. Methods: Using manuscripts from cancer randomized controlled trials recently published in high-tier clinical journals, we reconstructed patient-level data for overall survival (69 trials) and progression-free survival (54 trials). For each trial endpoint, we estimated the empirical power of each test. Empirical power was measured as the proportion of trials for which a test would have identified a significant result (p value < .05). Results: For overall survival, t-year event rate offered the lowest (30.4%) empirical power and restricted mean survival time with fixed τ offered the highest (43.5%). The empirical power of the other types of tests was almost identical (36.2%-37.7%). For progression-free survival, the tests we investigated offered numerically equivalent empirical power (55.6%-61.1%). No single test consistently outperformed any other test. Conclusion: The empirical power assessment with the past cancer randomized controlled trials provided new insights on the performance of statistical tests. Although the log-rank test has been used in almost all trials, our study suggests that the log-rank test is not the only option from an empirical power perspective. Near universal use of the log-rank test is not supported by a meaningful difference in empirical power. Clinical trial investigators could consider alternative methods, beyond the log-rank test, for their primary analysis when designing a cancer randomized controlled trial. Factors other than power (e.g. interpretability of the estimated treatment effect) should garner greater consideration when selecting statistical tests for cancer randomized controlled trials.

Tests of fit for the power function lognormal distribution

A power comparison of various normality tests

Comparative evaluation of goodness of fit tests for normal distribution using simulation and empirical data

Power analysis of several normality tests: A Monte Carlo simulation study

Comparison of selected tests for univariate normality based on measures of moments

Normal-Power-Logistic Distribution: Properties and Application in Generalized Linear Model

Size and power properties of some tests in the Birnbaum-Saunders regression model

Evaluating the Three Methods of Goodness of Fit Test for Frequency Analysis

Goodness-of-fit tests for modified Poisson regression possibly producing fitted values exceeding one in binary outcome analysis

Logistic or not Logistic?

Segmental estimation and testing method for power-law distributions and some extensions

A New Goodness-Of-Fit Test Based On The Laplace Statistic For A Large Class Of Nhpp Models

Empirical power comparison of statistical tests in contemporary phase III randomized controlled trials with time-to-event outcomes in oncology

Parametric Regression Model for Survival Data: Weibull Regression Model As an Example

Change Point Test for Length-Biased Lognormal Distribution under Random Right Censoring

More powerful logrank permutation tests for two-sample survival data

Goodness-offit tests and minimum power divergence estimators for survival data

Log-logistic Distribution as a Reliability Model: A Bayesian Analysis

The Tail Distribution of Fire Loss Data in Anhui Province

A comprehensive comparison of goodness-of-fit tests for logistic regression models

Goodness of Fit for the Waring Distribution