Abstract:<h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Background and Objective</h3><p>In recent years, an increasing number of clinical prediction models have been developed to serve clinical care. Establishing a data-driven prediction model based on large-scale electronic health record (EHR) data can provide a more empirical basis for clinical decision making. However, research on model generalization and continuous improvement is insufficiently focused, which also hinders the application and evaluation of prediction models in real clinical environments. Therefore, this study proposes a multicenter collaborative prediction model construction framework to build a prediction model with greater generalizability and continuous improvement capabilities while preserving patient data security and privacy.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Materials and Methods</h3><p>Based on a multicenter collaborative research network, such as the Observational Health Data Sciences and Informatics (OHDSI), a multicenter collaborative prediction model construction framework is proposed. Based on the idea of multi-source transfer learning, in each source hospital, a base classifier was trained according to the model research setting. Then, in the target hospital with missing calibration data, a prediction model was established through weighted integration of base classifiers from source hospitals based on the smoothness assumption. Moreover, a passive-aggressive online learning algorithm was used for continuous improvement of the prediction model, which can help to maintain a high predictive performance to provide reliable clinical decision-making abilities. To evaluate the proposed prediction model construction framework, a prototype system for colorectal cancer prognosis prediction was developed. To evaluate the performance of models, 70,906 patients were screened, including 70,090 from 5 US hospital-specific datasets and 816 from a Chinese hospital-specific dataset. The area under the receiver operating characteristic curve (AUC) and the estimated calibration index (ECI) were used to evaluate the discrimination and calibration of models.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Results</h3><p>Regarding the colorectal cancer prognosis prediction in our prototype system, compared with the reference models, our model achieved a better performance in model calibration (ECI = 9.294 [9.146, 9.441]) and a similar ability in model discrimination (AUC = 0.783 [0.780, 0.786]). Furthermore, the online learning process provided in this study can continuously improve the performance of the prediction model when patient data with specified labels arrive (the AUC value increased from 0.709 to 0.715 and the ECI value decreased from 13.013 to 9.634 after 650 patient instances with specified labels from the Chinese hospital arrived), enabling the prediction model to maintain a good predictive performance during clinical application.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Conclusions</h3><p>This study proposes and evaluates a multicenter collaborative prediction model construction framework that can support the construction of prediction models with better generalizability and continuous improvement capabilities without the need to aggregate multicenter patient-level data.</p>

A scalable approach for developing clinical risk prediction applications in different hospitals

Cost-sensitive Deep Learning for Early Readmission Prediction at A Major Hospital

Predicting Hospital Readmission Via Cost-Sensitive Deep Learning

Use of deep learning to develop continuous-risk models for adverse event prediction from electronic health records

Towards artificial intelligence-based disease prediction algorithms that comprehensively leverage and continuously learn from real-world clinical tabular data systems

Scalable and accurate deep learning with electronic health records

Scalable Predictive Analysis in Critically Ill Patients Using a Visual Open Data Analysis Platform

From Single-Hospital to Multi-Centre Applications: Enhancing the Generalisability of Deep Learning Models for Adverse Event Prediction in the ICU

Using Transfer Learning for Improved Mortality Prediction in a Data-Scarce Hospital Setting

Clinical Prediction Models for Hospital-Induced Delirium Using Structured and Unstructured Electronic Health Record Data: Protocol for a Development and Validation Study

Using machine learning techniques to predict hospital admission at the emergency department

A Knowledge Distillation Ensemble Framework for Predicting Short and Long-term Hospitalisation Outcomes from Electronic Health Records Data

Development and Validation of an Electronic Health Record–Based Machine Learning Model to Estimate Delirium Risk in Newly Hospitalized Patients Without Known Cognitive Impairment

In-hospital mortality, readmission, and prolonged length of stay risk prediction leveraging historical electronic patient records

Establishment and evaluation of a multicenter collaborative prediction model construction framework supporting model generalization and continuous improvement: A pilot study

Application of Machine Learning Techniques to Development of Emergency Medical Rapid Triage Prediction Models in Acute Care

Leveraging Clinical Time-Series Data for Prediction: A Cautionary Tale

Improving prediction for medical institution with limited patient data: Leveraging hospital-specific data based on multicenter collaborative research network

Development & Deployment of a Real-time Healthcare Predictive Analytics Platform

Transatlantic transferability and replicability of machine-learning algorithms to predict mental health crises

Distillation to Enhance the Portability of Risk Models Across Institutions with Large Patient Claims Database