Abstract:OBJECTIVE:Clinical prediction models (CPMs) constructed based on artificial intelligence have been proven to have positive impacts on clinical activities. However, the deterioration of CPM performance over time has rarely been studied. This paper proposes a model updating method to solve the calibration drift issue caused by data drift. MATERIALS AND METHODS:This paper proposes a novel model updating method based on lifelong machine learning (LML). The effectiveness of the proposed method is verified in four tumor datasets, and a comprehensive comparison with other model updating methods is performed. RESULTS:Changes in data distributions cause model performances to drift. The four compared model updating methods have different effects in terms of improving the discrimination and calibration abilities of the tested models. The LML method proposed in this study improves model performance better than or equivalent to the other methods. The proposed method achieved a mean AUC of 0.8249, 0.8780, 0.8261, and 0.8489, a mean AUPRC of 0.7782, 0.9730, 0.4655, and 0.5728, a mean F1 of 0.6866, 0.9552, 0.2985, and 0.3585, and a mean estimated calibration index (ECI) of 0.0320, 0.0338, 0.0101, and 0.0115 using colorectal, lung, breast and prostate cancer datasets. DISCUSSION:The LML framework simultaneously monitors model performance and the distribution of disease risk characteristics, enabling it to effectively address the performance degradation caused by gradual and sudden data drifts and provide reasonable explanations for the causes of performance degradation. CONCLUSION:Monitoring model performance and the underlying data distribution can promote model life cycle iteration with "development-deployment-maintenance-monitoring" as the core, which, in turn, ensures that the model can provide accurate predictions, guides the model update process and explains the causes of model performance changes.

A linear adjustment based approach to posterior drift in transfer learning

A novel lifelong machine learning-based method to eliminate calibration drift in clinical prediction models.

Application of concept drift detection and adaptive framework for non linear time series data from cardiac surgery

A penalized likelihood approach for efficiently estimating a partially linear additive transformation model with current status data

A hybrid adaptive approach for instance transfer learning with dynamic and imbalanced data

Automated, efficient and model-free inference for randomized clinical trials via data-driven covariate adjustment

Efficient and Multiply Robust Risk Estimation under General Forms of Dataset Shift

Dynamic Logistic Regression and Dynamic Model Averaging for Binary Classification

Regularized Bayesian transfer learning for population level etiological distributions

Bayesian Joint Model with Latent Time Shifts for Multivariate Longitudinal Data with Informative Dropout

Brownian Motion with Adaptive Drift for Remaining Useful Life Prediction: Revisited

An adaptive transfer learning perspective on classification in non-stationary environments

Joint Dynamic Models and Statistical Inference for Recurrent Competing Risks, Longitudinal Marker, and Health Status

Doubly Robust Augmented Model Accuracy Transfer Inference with High Dimensional Features

An Adaptive Algorithm for Learning with Unknown Distribution Drift

A Scalable Approach to Covariate and Concept Drift Management via Adaptive Data Segmentation

Bias correction for multiple covariate analysis using empirical bayesian estimation in mixed-effects models for longitudinal data

Model Migration Through Bayesian Adjustments

Model adaptation and unsupervised learning with non-stationary batch data under smooth concept drift

Handling Concept Drifts in Regression Problems -- the Error Intersection Approach

Bias Assessment and Data Drift Detection in Medical Image Analysis: A Survey