Abstract:Hospital readmission prediction is considered an essential approach to decreasing readmission rates, which is a key factor in assessing the quality and efficacy of a healthcare system. Previous studies have extensively utilized three primary modalities, namely electronic health records (EHR), medical images, and clinical notes, to predict hospital readmissions. However, the majority of these studies did not integrate information from all three modalities or utilize the spatiotemporal relationships present in the dataset. This study introduces a novel model called the Multimodal Spatiotemporal Graph-Transformer (MuST) for predicting hospital readmissions. By employing Graph Convolution Networks and temporal transformers, we can effectively capture spatial and temporal dependencies in EHR and chest radiographs. We then propose a fusion transformer to combine the spatiotemporal features from the two modalities mentioned above with the features from clinical notes extracted by a pre-trained, domain-specific transformer. We assess the effectiveness of our methods using the latest publicly available dataset, MIMIC-IV. The experimental results indicate that the inclusion of multimodal features in MuST improves its performance in comparison to unimodal methods. Furthermore, our proposed pipeline outperforms the current leading methods in the prediction of hospital readmissions.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is **hospital readmission prediction**. Specifically, the author aims to improve the prediction accuracy of whether a patient will be readmitted within 30 days after discharge by integrating three main medical data modalities (electronic health records (EHR), medical images, and clinical notes) using deep - learning techniques. ### Background and Problem Description 1. **Importance of Hospital Readmission Rates**: - Hospital readmission rates are one of the key indicators for evaluating the quality and efficiency of the medical system. - Reducing unnecessary readmission rates is the top priority of the medical system. 2. **Deficiencies in Existing Research**: - Most previous studies have only utilized part of the data in the three modalities or failed to fully utilize the spatio - temporal relationships in the data. - Most studies have not fully integrated the data of the three modalities: EHR, medical images, and clinical notes. ### Research Objectives - **Propose a new model**: Multimodal Spatiotemporal Graph - Transformer (MuST) for predicting hospital readmission. - **Integrate multi - modal data**: Capture the spatio - temporal dependencies in EHR and chest X - ray images through Graph Convolution Networks and Temporal Transformers. - **Fuse multi - modal features**: Introduce a Fusion Transformer to effectively fuse the features extracted from EHR, medical images, and clinical notes to generate more accurate readmission predictions. ### Method Overview 1. **Feature Extraction**: - For EHR and medical images, construct a graph structure and use graph neural networks to capture spatial dependencies. - Use pre - trained BioClinical BERT to extract meaningful representations from clinical notes. 2. **Spatio - Temporal Feature Learning**: - Capture the time - dependent features of each node through the Temporal Transformer. 3. **Multi - Modal Fusion**: - Use the Fusion Transformer to aggregate features from different modalities to generate the final prediction results. ### Experimental Verification - Extensive experiments were carried out on the publicly available MIMIC - IV dataset to verify the effectiveness and superiority of the MuST framework. - The experimental results show that MuST is significantly superior to existing single - modal and other multi - modal methods in evaluation metrics such as AUC and ACC. ### Conclusion By integrating multi - modal data and utilizing spatio - temporal relationships, the MuST model performs excellently in the hospital readmission prediction task, providing strong support for improving the efficiency and quality of the medical system.

MuST: Multimodal Spatiotemporal Graph-Transformer for Hospital Readmission Prediction

Cost-sensitive Deep Learning for Early Readmission Prediction at A Major Hospital

Predicting Hospital Readmission Via Cost-Sensitive Deep Learning

Multimodal spatiotemporal graph neural networks for improved prediction of 30-day all-cause hospital readmission

Static and multivariate-temporal attentive fusion transformer for readmission risk prediction

Multimodal risk prediction with physiological signals, medical images and clinical notes

Research on Multimodal Fusion of Temporal Electronic Medical Records

Predicting 30-Day Hospital Readmission in Medicare Patients: Insights from an LSTM Deep Learning Model

M3T-LM: A multi-modal multi-task learning model for jointly predicting patient length of stay and mortality

Multi-modal learning for inpatient length of stay prediction

Multimodal fusion network for ICU patient outcome prediction

Predicting unplanned readmissions in the intensive care unit: a multimodality evaluation

Multimodal Ensemble Approach to Incorporate Various Types of Clinical Notes for Predicting Readmission

Modeling Long-term Dependencies and Short-term Correlations in Patient Journey Data with Temporal Attention Networks for Health Prediction

How to Leverage Multimodal EHR Data for Better Medical Predictions?

Predicting Hospital Readmission: A Joint Ensemble-Learning Model.

Multimodal temporal-clinical note network for mortality prediction

Heart failure prognosis prediction : Let's start with the MDL-HFP model

Predicting ICU Readmission from Electronic Health Records via BERTopic with Long Short Term Memory Network Approach

Improving the Prognostic Evaluation Precision of Hospital Outcomes for Heart Failure Using Admission Notes and Clinical Tabular Data: Multimodal Deep Learning Model