Abstract:Summary Background Comorbidity, frailty, and decreased cognitive function lead to a higher risk of death in elderly patients (more than 65 years of age) during acute medical events. Early and accurate illness severity assessment can support appropriate decision making for clinicians caring for these patients. We aimed to develop ELDER-ICU, a machine learning model to assess the illness severity of older adults admitted to the intensive care unit (ICU) with cohort-specific calibration and evaluation for potential model bias. Methods In this retrospective, international multicentre study, the ELDER-ICU model was developed using data from 14 US hospitals, and validated in 171 hospitals from the USA and Netherlands. Data were extracted from the Medical Information Mart for Intensive Care database, electronic ICU Collaborative Research Database, and Amsterdam University Medical Centers Database. We used six categories of data as predictors, including demographics and comorbidities, physical frailty, laboratory tests, vital signs, treatments, and urine output. Patient data from the first day of ICU stay were used to predict in-hospital mortality. We used the eXtreme Gradient Boosting algorithm (XGBoost) to develop models and the SHapley Additive exPlanations method to explain model prediction. The trained model was calibrated before internal, external, and temporal validation. The final XGBoost model was compared against three other machine learning algorithms and five clinical scores. We performed subgroup analysis based on age, sex, and race. We assessed the discrimination and calibration of models using the area under receiver operating characteristic (AUROC) and standardised mortality ratio (SMR) with 95% CIs. Findings Using the development dataset (n=50 366) and predictive model building process, the XGBoost algorithm performed the best in all types of validations compared with other machine learning algorithms and clinical scores (internal validation with 5037 patients from 14 US hospitals, AUROC=0·866 [95% CI 0·851–0·880]; external validation in the US population with 20 541 patients from 169 hospitals, AUROC=0·838 [0·829–0·847]; external validation in European population with 2411 patients from one hospital, AUROC=0·833 [0·812–0·853]; temporal validation with 4311 patients from one hospital, AUROC=0·884 [0·869–0·897]). In the external validation set (US population), the median AUROCs of bias evaluations covering eight subgroups were above 0·81, and the overall SMR was 0·99 (0·96–1·03). The top ten risk predictors were the minimum Glasgow Coma Scale score, total urine output, average respiratory rate, mechanical ventilation use, best state of activity, Charlson Comorbidity Index score, geriatric nutritional risk index, code status, age, and maximum blood urea nitrogen. A simplified model containing only the top 20 features (ELDER-ICU-20) had similar predictive performance to the full model. Interpretation The ELDER-ICU model reliably predicts the risk of in-hospital mortality using routinely collected clinical features. The predictions could inform clinicians about patients who are at elevated risk of deterioration. Prospective validation of this model in clinical practice and a process for continuous performance monitoring and model recalibration are needed. Funding National Institutes of Health, National Natural Science Foundation of China, National Special Health Science Program, Health Science and Technology Plan of Zhejiang Province, Fundamental Research Funds for the Central Universities, Drug Clinical Evaluate Research of Chinese Pharmaceutical Association, and National Key R&D Program of China.

Development, External Validation, and Biomolecular Corroboration of Interoperable Models for Identifying Critically Ill Children at Risk of Neurologic Morbidity

Pediatric severe traumatic brain injury mortality prediction determined with machine learning-based modeling

NEUROlogical Prognosis After Cardiac Arrest in Kids (NEUROPACK) study: protocol for a prospective multicentre clinical prediction model derivation and validation study in children after cardiac arrest

Early identification of patients admitted to hospital for covid-19 at risk of clinical deterioration: model development and multisite external validation study

Machine learning-based prediction of delirium 24 h after pediatric intensive care unit admission in critically ill children:A prospective cohort study

Early Prediction of Multiple Organ Dysfunction in the Pediatric Intensive Care Unit

Development of machine learning models predicting mortality using routinely collected observational health data from 0-59 months old children admitted to an intensive care unit in Bangladesh: critical role of biochemistry and haematology data

Geographical validation of the Smart Triage Model by age group

A Data-Driven Algorithm Integrating Clinical and Laboratory Features for the Diagnosis and Prognosis of Necrotizing Enterocolitis

Machine Learning Model for Predicting Mortality Risk in Pediatric Cerebral Venous Sinus Thrombosis

Development and Internal Validation of a Prediction Model to Risk Stratify Children With Suspected Community-Acquired Pneumonia

Development and validation of a diagnostic model for early differentiation of sepsis and non-infectious SIRS in critically ill children - a data-driven approach using machine-learning algorithms

Elicitation of domain knowledge for a machine learning model for paediatric critical illness in South Africa

Prospective clinical testing and experimental validation of the Pediatric Sepsis Biomarker Risk Model

Machine learning-based prediction of critical illness in children visiting the emergency department

Developing machine learning models for prediction of mortality in the medical intensive care unit

Illness severity assessment of older adults in critical illness using machine learning (ELDER-ICU): an international multicentre study with subgroup bias evaluation

Machine Learning-Based Prediction of Clinical Outcomes for Children During Emergency Department Triage

Identification and validation of an explainable prediction model of acute kidney injury with prognostic implications in critically ill children: a prospective multicenter cohort study

Predictive Models of Neurodevelopmental Outcomes After Neonatal Hypoxic-Ischemic Encephalopathy

Improving ICU Risk Predictive Models Through Automation Designed for Resiliency Against Documentation Bias