Static and multivariate-temporal attentive fusion transformer for readmission risk prediction

Zhe Sun,Runzhi Li,Jing Wang,Gang Chen,Siyu Yan,Lihong Ma
2024-07-15
Abstract:Background: Accurate short-term readmission prediction of ICU patients is significant in improving the efficiency of resource assignment by assisting physicians in making discharge decisions. Clinically, both individual static static and multivariate temporal data collected from ICU monitors play critical roles in short-term readmission prediction. Informative static and multivariate temporal feature representation capturing and fusion present challenges for accurate readmission prediction. Methods:We propose a novel static and multivariate-temporal attentive fusion transformer (SMTAFormer) to predict short-term readmission of ICU patients by fully leveraging the potential of demographic and dynamic temporal data. In SMTAFormer, we first apply an MLP network and a temporal transformer network to learn useful static and temporal feature representations, respectively. Then, the well-designed static and multivariate temporal feature fusion module is applied to fuse static and temporal feature representations by modeling intra-correlation among multivariate temporal features and constructing inter-correlation between static and multivariate temporal features. Results: We construct a readmission risk assessment (RRA) dataset based on the MIMIC-III dataset. The extensive experiments show that SMTAFormer outperforms advanced methods, in which the accuracy of our proposed method is up to 86.6%, and the area under the receiver operating characteristic curve (AUC) is up to 0.717. Conclusion: Our proposed SMTAFormer can efficiently capture and fuse static and multivariate temporal feature representations. The results show that SMTAFormer significantly improves the short-term readmission prediction performance of ICU patients through comparisons to strong baselines.
Machine Learning,Artificial Intelligence
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: **How to accurately predict the short - term readmission risk of ICU (Intensive Care Unit) patients**. Specifically, the author aims to improve the prediction accuracy of the readmission risk of ICU patients within 30 days after discharge by making full use of static data (such as demographic information) and multivariate time - series data (such as physiological monitoring data). ### Problem Background 1. **Clinical Requirement**: The short - term readmission of ICU patients not only prolongs the length of hospital stay but also increases the risk of death. Therefore, accurately predicting the readmission risk is crucial for optimizing the allocation of medical resources and assisting doctors in making reasonable discharge decisions. 2. **Existing Challenges**: Existing methods have deficiencies in dealing with static data and multivariate time - series data, and it is difficult to effectively capture and fuse the key features in these two types of data, thus affecting the prediction performance. ### Solution To solve the above problems, the author proposes a new deep - learning model - **Static and Multivariate Time - series Attention Fusion Transformer (SMTAFormer)**. The main innovations of this model include: - **Static Feature Extraction**: Use the MLP (Multi - Layer Perceptron) network to learn useful feature representations from static data. - **Time - series Feature Extraction**: Utilize the Transformer encoder to learn dynamic feature representations from multivariate time - series data. - **Feature Fusion Module**: Design a novel static and multivariate time - series feature fusion module, which realizes the dynamic fusion of the two by modeling the internal correlation between multivariate time - series features and the external correlation between static and multivariate time - series features. ### Experimental Results Through extensive experiments on the RRA (Readmission Risk Assessment) dataset constructed based on the MIMIC - III dataset, SMTAFormer significantly outperforms existing methods in multiple evaluation metrics, with an accuracy rate reaching 86.6% and an AUC (Area Under the Receiver Operating Characteristic Curve) reaching 0.717. ### Summary This research effectively solves the fusion problem of static and multivariate time - series data in predicting the short - term readmission risk of ICU patients by introducing the SMTAFormer model, providing strong support for clinical decision - making. Future research will further expand the feature set and explore the possibility of applying this method to local readmission datasets. ### Formula Summary The formulas involved in this paper mainly include: - Static Feature Extraction Formula: \[ \mathbf{S}=\text{ReLU}(\mathbf{S}W_s + b_s) \] where \(\mathbf{S}\) is the generated static feature representation, and \(W_s\) and \(b_s\) are learnable parameters. - Time - series Feature Extraction Formula: \[ E_i=\text{ReLU}(X_iW_t^i + b_t^i) \] \[ P_i = P+E_i \] \[ M_i=\text{Concat}(\text{head}_1,\text{head}_2,\dots,\text{head}_h)W \] \[ \text{head}_l=\text{Attention}(P_iW_Q^l,P_iW_K^l,P_iW_V^l) \] \[ \text{Attention}(Q, K, V)=\text{softmax}\left(\frac{QK^T}{\sqrt{d}}\right)V \] - Feature Fusion Formula: \[ M=\text{Concat}(\text{head}'_1,\text{head}'_2,\dots,\text{head}'_h)\tilde{W} \] \[ \tilde{X}=\text{Concat}(\text{head}''_1,\text{head}''_2,\dots,\text{head}''_h)W \]