Abstract:Background: Over time, the performance of clinical prediction models may deteriorate due to changes in clinical management, data quality, disease risk and/or patient mix. Such prediction models must be updated in order to remain useful. In this study, we investigate dynamic model updating of clinical survival prediction models. In contrast to discrete or one-time updating, dynamic updating refers to a repeated process for updating a prediction model with new data. We aim to extend previous research which focused largely on binary outcome prediction models by concentrating on time-to-event outcomes. We were motivated by the rapidly changing environment seen during the COVID-19 pandemic where mortality rates changed over time and new treatments and vaccines were introduced. Methods: We illustrate three methods for dynamic model updating: Bayesian dynamic updating, recalibration, and full refitting. We use a simulation study to compare performance in a range of scenarios including changing mortality rates, predictors with low prevalence and the introduction of a new treatment. Next, the updating strategies were applied to a model for predicting 70-day COVID-19-related mortality using patient data from QResearch, an electronic health records database from general practices in the UK. Results: In simulated scenarios with mortality rates changing over time, all updating methods resulted in better calibration than not updating. Moreover, dynamic updating outperformed ad hoc updating. In the simulation scenario with a new predictor and a small updating dataset, Bayesian updating improved the C-index over not updating and refitting. In the motivating example with a rare outcome, no single updating method offered the best performance. Conclusions: We found that a dynamic updating process outperformed one-time discrete updating in the simulations. Bayesian updating offered good performance overall, even in scenarios with new predictors and few events. Intercept recalibration was effective in scenarios with smaller sample size and changing baseline hazard. Refitting performance depended on sample size and produced abrupt changes in hazard ratio estimates between periods.

[Methodology and progress in adjusting time-dependent covariates in clinical prediction models]

A novel lifelong machine learning-based method to eliminate calibration drift in clinical prediction models.

Extracting Dynamic Information of Temporal Clinical Data to Predict the Outcome in Critically Ill Patients.

Harnessing repeated measurements of predictor variables for clinical risk prediction: a review of existing methods

Predictive analytics with gradient boosting in clinical medicine

Time-varying Covariates and Coefficients in Cox Regression Models

Reinforcement Learning in Clinical Medicine: a Method to Optimize Dynamic Treatment Regime over Time.

Landmark linear transformation model for dynamic prediction with application to a longitudinal cohort study of chronic disease

Automated, efficient and model-free inference for randomized clinical trials via data-driven covariate adjustment

Delayed kernels for longitudinal survival analysis and dynamic prediction

Predictive modeling of biomedical temporal data in healthcare applications: review and future directions

Addressing subject heterogeneity in time‐dependent discrimination for biomarker evaluation

Dynamic updating of clinical survival prediction models in a changing environment

Leveraging Clinical Time-Series Data for Prediction: A Cautionary Tale

Risk of Mortality Prediction Involving Time-Varying Covariates for Patients with Heart Failure Using Deep Learning

Estimating Discontinuous Time-Varying Risk Factors and Treatment Benefits for COVID-19 with Interpretable ML

Time-varying effect in older patients with early-stage breast cancer: a model considering the competing risks based on a time scale

Restricted mean survival time regression model with time-dependent covariates

Conformal Prediction Intervals with Temporal Dependence

Integrating Machine Learning into Statistical Methods in Disease Risk Prediction Modeling: A Systematic Review

[Introduction of Landmarking Approach and Its Application in Dynamic Prediction].