Abstract:Abstract Background We suggest an adaptive sample size calculation method for developing clinical prediction models, in which model performance is monitored sequentially as new data comes in. Methods We illustrate the approach using data for the diagnosis of ovarian cancer ( n = 5914, 33% event fraction) and obstructive coronary artery disease (CAD; n = 4888, 44% event fraction). We used logistic regression to develop a prediction model consisting only of a priori selected predictors and assumed linear relations for continuous predictors. We mimicked prospective patient recruitment by developing the model on 100 randomly selected patients, and we used bootstrapping to internally validate the model. We sequentially added 50 random new patients until we reached a sample size of 3000 and re-estimated model performance at each step. We examined the required sample size for satisfying the following stopping rule: obtaining a calibration slope ≥ 0.9 and optimism in the c-statistic (or AUC) < = 0.02 at two consecutive sample sizes. This procedure was repeated 500 times. We also investigated the impact of alternative modeling strategies: modeling nonlinear relations for continuous predictors and correcting for bias on the model estimates (Firth’s correction). Results Better discrimination was achieved in the ovarian cancer data (c-statistic 0.9 with 7 predictors) than in the CAD data (c-statistic 0.7 with 11 predictors). Adequate calibration and limited optimism in discrimination was achieved after a median of 450 patients (interquartile range 450–500) for the ovarian cancer data (22 events per parameter (EPP), 20–24) and 850 patients (750–900) for the CAD data (33 EPP, 30–35). A stricter criterion, requiring AUC optimism < = 0.01, was met with a median of 500 (23 EPP) and 1500 (59 EPP) patients, respectively. These sample sizes were much higher than the well-known 10 EPP rule of thumb and slightly higher than a recently published fixed sample size calculation method by Riley et al. Higher sample sizes were required when nonlinear relationships were modeled, and lower sample sizes when Firth’s correction was used. Conclusions Adaptive sample size determination can be a useful supplement to fixed a priori sample size calculations, because it allows to tailor the sample size to the specific prediction modeling context in a dynamic fashion.

Extension of a conditional performance score for sample size recalculation rules to the setting of binary endpoints

Smoothing Corrections for Improving Sample Size Recalculation Rules in Adaptive Group Sequential Study Designs

Conditional Power and Friends: The Why and How of (Un)planned, Unblinded Sample Size Recalculations in Confirmatory Trials

Sample size recalculation in three-stage clinical trials and its evaluation

Adaptive clinical trial designs with blinded selection of binary composite endpoints and sample size reassessment

Sample size re-estimation in a superiority clinical trial using a hybrid classical and Bayesian procedure

A new approach for sizing trials with composite binary endpoints using anticipated marginal values and accounting for the correlation between components

Sample Size Calculation and Blinded Sample Size Recalculation in Clinical Trials Where the Treatment Effect is Measured by the Relative Risk

The Conditional Power of Interim Analysis and Sample Size Re-estimation

Design and monitoring of clinical trials with an interim analysis and a negative binomial endpoint

SMART Binary: Sample Size Calculation for Comparing Adaptive Interventions in SMART studies with Longitudinal Binary Outcomes

Power/sample size calculations for assessing correlates of risk in clinical efficacy trials

Design of randomized clinical trials with a binary endpoint: Conditional versus unconditional analyses of a two‐by‐two table

Optimal sample size allocation for two‐arm superiority and non‐inferiority trials with binary endpoints

Estimating the Sampling Distribution of Posterior Decision Summaries in Bayesian Clinical Trials

Sample Size Determination and Re‐estimation for Matched Pair Designs with Multiple Binary Endpoints

Sample Size Calculation for “Gold-Standard” Noninferiority Trials With Fixed Margins and Negative Binomial Endpoints

Estimation of required sample size for external validation of risk models for binary outcomes

Adaptive sample size determination for the development of clinical prediction models

Reporting and communication of sample size calculations in adaptive clinical trials: a review of trial protocols and grant applications

Simplification or simulation: Power calculation in clinical trials