Abstract:Abstract Background Accurate, pragmatic risk stratification for postoperative delirium (POD) is necessary to target preventative resources toward high-risk patients. Machine learning (ML) offers a novel approach to leveraging electronic health record (EHR) data for POD prediction. We sought to develop and internally validate a ML-derived POD risk prediction model using preoperative risk features, and to compare its performance to models developed with traditional logistic regression. Methods This was a retrospective analysis of preoperative EHR data from 24,885 adults undergoing a procedure requiring anesthesia care, recovering in the main post-anesthesia care unit, and staying in the hospital at least overnight between December 2016 and December 2019 at either of two hospitals in a tertiary care health system. One hundred fifteen preoperative risk features including demographics, comorbidities, nursing assessments, surgery type, and other preoperative EHR data were used to predict postoperative delirium (POD), defined as any instance of Nursing Delirium Screening Scale ≥2 or positive Confusion Assessment Method for the Intensive Care Unit within the first 7 postoperative days. Two ML models (Neural Network and XGBoost), two traditional logistic regression models (“clinician-guided” and “ML hybrid”), and a previously described delirium risk stratification tool (AWOL-S) were evaluated using the area under the receiver operating characteristic curve (AUC-ROC), sensitivity, specificity, positive likelihood ratio, and positive predictive value. Model calibration was assessed with a calibration curve. Patients with no POD assessments charted or at least 20% of input variables missing were excluded. Results POD incidence was 5.3%. The AUC-ROC for Neural Net was 0.841 [95% CI 0. 816–0.863] and for XGBoost was 0.851 [95% CI 0.827–0.874], which was significantly better than the clinician-guided (AUC-ROC 0.763 [0.734–0.793], p < 0.001) and ML hybrid (AUC-ROC 0.824 [0.800–0.849], p < 0.001) regression models and AWOL-S (AUC-ROC 0.762 [95% CI 0.713–0.812], p < 0.001). Neural Net, XGBoost, and ML hybrid models demonstrated excellent calibration, while calibration of the clinician-guided and AWOL-S models was moderate; they tended to overestimate delirium risk in those already at highest risk. Conclusion Using pragmatically collected EHR data, two ML models predicted POD in a broad perioperative population with high discrimination. Optimal application of the models would provide automated, real-time delirium risk stratification to improve perioperative management of surgical patients at risk for POD.

A machine learning-based prediction model for postoperative delirium in cardiac valve surgery using electronic health records

Research progress in the Delirium Risk Prediction Models of post-operative patients in ICU

Risk Predictive Models for Delirium in the Intensive Care Unit: a Systematic Review and Meta-Analysis.

Postoperative delirium prediction after cardiac surgery using machine learning models

Validation of E-PRE-DELIRIC in cardiac surgical ICU delirium: A retrospective cohort study

Machine learning-based prediction model for postoperative delirium in non-cardiac surgery

Automated machine learning‐based model predicts postoperative delirium using readily extractable perioperative collected electronic data

Machine Learning Models of Postoperative Atrial Fibrillation Prediction After Cardiac Surgery.

Postoperative delirium prediction using machine learning models and preoperative electronic health record data

Machine learning with clinical and intraoperative biosignal data for predicting postoperative delirium after cardiac surgery

Development of postoperative delirium prediction models in patients undergoing cardiovascular surgery using machine learning algorithms

Machine Learning for Prediction of Postoperative Delirium in Adult Patients: A Systematic Review and Meta-analysis

A machine-learning model to predict postoperative delirium following knee arthroplasty using electronic health records

Machine Learning Algorithm Using Electronic Chart-Derived Data to Predict Delirium After Elderly Hip Fracture Surgeries: A Retrospective Case-Control Study

Predicting postoperative delirium after hip arthroplasty for elderly patients using machine learning

Two-stage prediction model for postoperative delirium in patients in the intensive care unit after cardiac surgery

Development and validation of a postoperative delirium prediction model for patients admitted to an intensive care unit in China: a prospective study

Machine learning methods for developing a predictive model of the incidence of delirium in cardiac intensive care units

Development and Validation of Machine Learning Models to Predict Postoperative Delirium Using Clinical Features and Polysomnography Variables

Prediction models for postoperative delirium after cardiac surgery: Systematic review and critical appraisal

Machine learning-based delirium prediction in surgical in-patients: a prospective validation study