Abstract:BackgroundMultiple primary outcomes may be specified in randomised controlled trials (RCTs). When analysing multiple outcomes it's important to control the family wise error rate (FWER). A popular approach to do this is to adjust the p-values corresponding to each statistical test used to investigate the intervention effects by using the Bonferroni correction. It's also important to consider the power of the trial to detect true intervention effects. In the context of multiple outcomes, depending on the clinical objective, the power can be defined as: 'disjunctive power', the probability of detecting at least one true intervention effect across all the outcomes or 'marginal power' the probability of finding a true intervention effect on a nominated outcome.We provide practical recommendations on which method may be used to adjust for multiple comparisons in the sample size calculation and the analysis of RCTs with multiple primary outcomes. We also discuss the implications on the sample size for obtaining 90% disjunctive power and 90% marginal power.MethodsWe use simulation studies to investigate the disjunctive power, marginal power and FWER obtained after applying Bonferroni, Holm, Hochberg, Dubey/Armitage-Parmar and Stepdown-minP adjustment methods. Different simulation scenarios were constructed by varying the number of outcomes, degree of correlation between the outcomes, intervention effect sizes and proportion of missing data.ResultsThe Bonferroni and Holm methods provide the same disjunctive power. The Hochberg and Hommel methods provide power gains for the analysis, albeit small, in comparison to the Bonferroni method. The Stepdown-minP procedure performs well for complete data. However, it removes participants with missing values prior to the analysis resulting in a loss of power when there are missing data. The sample size requirement to achieve the desired disjunctive power may be smaller than that required to achieve the desired marginal power. The choice between whether to specify a disjunctive or marginal power should depend on the clincial objective.

Some statistical methods for multiple endpoints in clinical trials

Multivariate Rank-Based Analysis of Multiple Endpoints in Clinical Trials: A Global Test Approach

A Nonparametric Global Win Probability Approach to the Analysis and Sizing of Randomized Controlled Trials With Multiple Endpoints of Different Scales and Missing Data: Beyond O'Brien–Wei–Lachin

Methods to adjust for multiple comparisons in the analysis and sample size calculation of randomised controlled trials with multiple primary outcomes

Optimal exact tests for multiple binary endpoints

Exploring Statistical Methods in Clinical Trials When Some of the Patients are Treated against the Randomization

Overview of constructing methods for the composite endpoint evaluation index in clinical trials

Multiple random change points in survival analysis with applications to clinical trials

Sample Size Calculation for Single-Arm OPC Trials with Multi Endpoints

A Bayesian Platform Trial Design to Simultaneously Evaluate Multiple Drugs in Multiple Indications with Mixed Endpoints.

Alternative Analysis Methods for Time to Event Endpoints under Non-proportional Hazards: A Comparative Analysis

The impact of allocation bias on test decisions in clinical trials with multiple endpoints using multiple testing strategies

Comparisons of methods for multiple hypothesis testing in neuropsychological research.

Multi-center clinical trials: Randomization and ancillary statistics

Clinician's Approach to Advanced Statistical Methods: Win Ratios, Restricted Mean Survival Time, Responder Analyses, and Standardized Mean Differences

Evaluating multiplicity reporting in analgesic clinical trials: An analytical review

Practical Recommendations for Regional Consistency Evaluation in Multi-Regional Clinical Trials with Different Endpoints

Statistical considerations for outcomes in clinical research: A review of common data types and methodology

Quantitative Safety Monitoring in Clinical Trials: Application of Multiple Statistical Methodologies for Infrequent Events

Alternative Analysis Methods for Time to Event Endpoints Under Nonproportional Hazards: A Comparative Analysis

New strategies for confirmatory testing of secondary hypotheses on combined data from multiple trials