Abstract:Title: A Systematic Review of Machine Learning-based Prognostic Models for Acute Pancreatitis: Towards Improving Methods and Reporting Quality Background: An accurate prognostic tool is essential to aid clinical decision making (e.g., patient triage) and to advance personalized medicine. However, such prognostic tool is lacking for acute pancreatitis (AP). Increasingly machine learning (ML) techniques are being used to develop high-performing prognostic models in AP. However, methodologic and reporting quality has received little attention. High-quality reporting and study methodology are critical to model validity, reproducibility, and clinical implementation. In collaboration with content experts in ML methodology, we performed a systematic review critically appraising the quality of methodology and reporting of recently published ML AP prognostic models. Methods: Using a validated search strategy, we identified ML AP studies from the databases MEDLINE, PubMed, and EMBASE published between January 2021 and December 2023. Eligibility criteria included all retrospective or prospective studies that developed or validated new or existing ML models in patients with AP that predicted an outcome following an episode of AP. Meta-analysis was considered if there was homogeneity in the study design and in the type of outcome predicted. For risk of bias (ROB) assessment, we used the Prediction Model Risk of Bias Assessment Tool (PROBAST). Quality of reporting was assessed using the Transparent Reporting of a Multivariable Prediction Model of Individual Prognosis or Diagnosis – Artificial Intelligence (TRIPOD+AI) statement that defines standards for 27 items that should be reported in publications using ML prognostic models. Results: The search strategy identified 6480 publications of which 30 met the eligibility criteria. Studies originated from China (22), U.S (4), and other (4). All 30 studies developed a new ML model and none sought to validate an existing ML model, producing a total of 39 new ML models. AP severity (23/39) or mortality (6/39) were the most common outcomes predicted. The mean area-under-the-curve for all models and endpoints was 0.91 (SD 0.08). The ROB was high for at least one domain in all 39 models, particularly for the analysis domain (37/39 models). Steps were not taken to minimize over-optimistic model performance in 27/39 models. Due to heterogeneity in the study design and in how the outcomes were defined and determined, meta-analysis was not performed. Studies reported on only 15/27 items from TRIPOD+AI standards, with only 7/30 justifying sample size and 13/30 assessing data quality. Other reporting deficiencies included omissions regarding human-AI interaction (28/30), handling low-quality or incomplete data in practice (27/30), sharing analytical codes (25/30), study protocols (25/30) and reporting source data (19/30),. Discussion: There are significant deficiencies in the methodology and reporting of recently published ML based prognostic models in AP patients. These undermine the validity, reproducibility and implementation of these prognostic models despite their promise of superior predictive accuracy. Funding: none Registration: Research Registry (reviewregistry1727)

Research advances in machine learning models for acute pancreatitis

Prediction of the Severity of Acute Pancreatitis Using Machine Learning Models.

Machine Learning Predictive Models for Acute Pancreatitis: A Systematic Review.

Machine Learning Improves Prediction of Severity and Outcomes of Acute Pancreatitis: a Prospective Multi-Center Cohort Study

Automated Machine Learning for the Early Prediction of the Severity of Acute Pancreatitis in Hospitals

Comparison of MPL-ANN and PLS-DA models for predicting the severity of patients with acute pancreatitis: An exploratory study

Machine Learning Models of Acute Kidney Injury Prediction in Acute Pancreatitis Patients.

Early Prediction of Severe Acute Pancreatitis Using Machine Learning

Machine learning model identifies aggressive acute pancreatitis within 48 h of admission: a large retrospective study

Advances in Related Indicators in Evaluating the Severity and Prognosis of Acute Pancreatitis

Predicting the Need for Therapeutic Intervention and Mortality in Acute Pancreatitis: A Two-Center International Study Using Machine Learning

Development and Evaluation of Machine Learning Models and Nomogram for the Prediction of Severe Acute Pancreatitis

Interpretable prediction of 30-day mortality in patients with acute pancreatitis based on machine learning and SHAP

Prediction of Severe Acute Pancreatitis at a Very Early Stage of the Disease Using Artificial Intelligence Techniques, Without Laboratory Data or Imaging Tests: The PANCREATIA Study

A machine learning model for predicting acute kidney injury secondary to severe acute pancreatitis

Acute pancreatitis: A review of diagnosis, severity prediction and prognosis assessment from imaging technology, scoring system and artificial intelligence

Predictive model of acute kidney injury in critically ill patients with acute pancreatitis: a machine learning approach using the MIMIC-IV database

A Systematic Review of Machine Learning-based Prognostic Models for Acute Pancreatitis: Towards Improving Methods and Reporting Quality

Prediction of In-Hospital Mortality of Intensive Care Unit Patients with Acute Pancreatitis Based on an Explainable Machine Learning Algorithm.

Machine Learning for the Prediction of Acute Kidney Injury in Patients with Acute Pancreatitis Admitted to the Intensive Care Unit.

Automated machine learning for early prediction of acute kidney injury in acute pancreatitis