Abstract:Background During a pandemic, it is important for clinicians to stratify patients and decide who receives limited medical resources. Machine learning models have been proposed to accurately predict COVID-19 disease severity. Previous studies have typically tested only one machine learning algorithm and limited performance evaluation to area under the curve analysis. To obtain the best results possible, it may be important to test different machine learning algorithms to find the best prediction model. Objective In this study, we aimed to use automated machine learning (autoML) to train various machine learning algorithms. We selected the model that best predicted patients’ chances of surviving a SARS-CoV-2 infection. In addition, we identified which variables (ie, vital signs, biomarkers, comorbidities, etc) were the most influential in generating an accurate model. Methods Data were retrospectively collected from all patients who tested positive for COVID-19 at our institution between March 1 and July 3, 2020. We collected 48 variables from each patient within 36 hours before or after the index time (ie, real-time polymerase chain reaction positivity). Patients were followed for 30 days or until death. Patients’ data were used to build 20 machine learning models with various algorithms via autoML. The performance of machine learning models was measured by analyzing the area under the precision-recall curve (AUPCR). Subsequently, we established model interpretability via Shapley additive explanation and partial dependence plots to identify and rank variables that drove model predictions. Afterward, we conducted dimensionality reduction to extract the 10 most influential variables. AutoML models were retrained by only using these 10 variables, and the output models were evaluated against the model that used 48 variables. Results Data from 4313 patients were used to develop the models. The best model that was generated by using autoML and 48 variables was the stacked ensemble model (AUPRC=0.807). The two best independent models were the gradient boost machine and extreme gradient boost models, which had an AUPRC of 0.803 and 0.793, respectively. The deep learning model (AUPRC=0.73) was substantially inferior to the other models. The 10 most influential variables for generating high-performing models were systolic and diastolic blood pressure, age, pulse oximetry level, blood urea nitrogen level, lactate dehydrogenase level, D-dimer level, troponin level, respiratory rate, and Charlson comorbidity score. After the autoML models were retrained with these 10 variables, the stacked ensemble model still had the best performance (AUPRC=0.791). Conclusions We used autoML to develop high-performing models that predicted the survival of patients with COVID-19. In addition, we identified important variables that correlated with mortality. This is proof of concept that autoML is an efficient, effective, and informative method for generating machine learning–based clinical decision support tools.

Using Machine Learning Algorithms Based on Patient Admission Laboratory Parameters to Predict Adverse Outcomes in COVID-19 Patients

A prediction model based on machine learning for diagnosing the early COVID-19 patients

Dynamic Prognosis Model for Predicting Survival in Severe and Critically Ill COVID-19 Patients Using Machine Learning

Early Prediction of Mortality Risk among Patients with Severe COVID-19, Using Machine Learning

A Prediction Model for Clinical Outcomes of COVID-19 Hospitalized Patients: Construction and Accuracy Assessment

A Predictive Nomogram for Predicting Improved Clinical Outcome Probability in Patients with COVID-19 in Zhejiang Province, China

Machine Learning Based Clinical Decision Support System for Early COVID-19 Mortality Prediction

Using machine learning in prediction of ICU admission, mortality, and length of stay in the early stage of admission of COVID-19 patients

Unveiling the future of COVID-19 patient care: groundbreaking prediction models for severe outcomes or mortality in hospitalized cases

A robust and parsimonious machine learning method to predict ICU admission of COVID-19 patients

Early and fair COVID-19 outcome risk assessment using robust feature selection

An Early Warning Tool for Predicting Mortality Risk of COVID-19 Patients Using Machine Learning

Algorithms for predicting COVID outcome using ready-to-use laboratorial and clinical data

Predicting Clinical Outcomes in COVID-19 and Pneumonia Patients: A Machine Learning Approach

Statistical Analysis and Machine Learning Prediction of Disease Outcomes for COVID-19 and Pneumonia Patients

Using Automated Machine Learning to Predict the Mortality of Patients With COVID-19: Prediction Model Development Study

Predictive Models of Mortality for Hospitalized Patients With COVID-19: Retrospective Cohort Study

Clinical and inflammatory features based machine learning model for fatal risk prediction of hospitalized COVID-19 patients: results from a retrospective cohort study

Prediction of Disease Progression of COVID-19 Based on Machine Learning: A Retrospective Multicentre Cohort Study in Wuhan, China

Development of a COVID-19 early risk assessment system based on multiple machine learning algorithms and routine blood tests: a real-world study