Abstract:Objectives: To develop and validate a novel interpretable artificial intelligence (AI) model that integrates radiomic features, deep learning features, and imaging features at multiple semantic levels to predict the prognosis of intracerebral hemorrhage (ICH) patients at 6 months post-onset. Materials and methods: Retrospectively enrolled 222 patients with ICH for Non-contrast Computed Tomography (NCCT) images and clinical data, who were divided into a training cohort (n = 186, medical center 1) and an external testing cohort (n = 36, medical center 2). Following image preprocessing, the entire hematoma region was segmented by two radiologists as the volume of interest (VOI). Pyradiomics algorithm library was utilized to extract 1762 radiomics features, while a deep convolutional neural network (EfficientnetV2-L) was employed to extract 1000 deep learning features. Additionally, radiologists evaluated imaging features. Based on the three different modalities of features mentioned above, the Random Forest (RF) model was trained, resulting in three models (Radiomics Model, Radiomics-Clinical Model, and DL-Radiomics-Clinical Model). The performance and clinical utility of the models were assessed using the Area Under the Receiver Operating Characteristic Curve (AUC), calibration curve, and Decision Curve Analysis (DCA), with AUC compared using the DeLong test. Furthermore, this study employs three methods, Shapley Additive Explanations (SHAP), Grad-CAM, and Guided Grad-CAM, to conduct a multidimensional interpretability analysis of model decisions. Results: The Radiomics-Clinical Model and DL-Radiomics-Clinical Model exhibited relatively good predictive performance, with an AUC of 0.86 [95% Confidence Intervals (CI): 0.71, 0.95; P < 0.01] and 0.89 (95% CI: 0.74, 0.97; P < 0.01), respectively, in the external testing cohort. Conclusion: The multimodal explainable AI model proposed in this study can accurately predict the prognosis of ICH. Interpretability methods such as SHAP, Grad-CAM, and Guided Grad-Cam partially address the interpretability limitations of AI models. Integrating multimodal imaging features can effectively improve the performance of the model. Clinical relevance statement: Predicting the prognosis of patients with ICH is a key objective in emergency care. Accurate and efficient prognostic tools can effectively prevent, manage, and monitor adverse events in ICH patients, maximizing treatment outcomes.

Unified ICH Quantification and Prognosis Prediction in NCCT Images Using a Multi-Task Interpretable Network

Combining Convolutional and Recurrent Neural Networks for Classification of Focal Liver Lesions in Multi-phase CT Images.

Improved performance and robustness of multi-task representation learning with consistency loss between pretexts for intracranial hemorrhage identification in head CT

Intracerebral Hemorrhage Image Segmentation and Classification Based on Multi-task Learning of Shared Shallow Parameters

An interpretable artificial intelligence model based on CT for prognosis of intracerebral hemorrhage: a multicenter study

ICH-SCNet: Intracerebral Hemorrhage Segmentation and Prognosis Classification Network Using CLIP-guided SAM mechanism

Intracerebral Hemorrhage Prognosis Classification via Joint-Attention Cross-Modal Network

Segmentation of Infarct Lesions and Prognosis Prediction for Acute Ischemic Stroke Using Non-Contrast CT Scans

Predicting Hematoma Expansion in Intracerebral Hemorrhage from Brain CT Scans Via K-nearest Neighbors Matting and Deep Residual Network.

ICHPro: Intracerebral Hemorrhage Prognosis Classification Via Joint-attention Fusion-based 3d Cross-modal Network

End-to-End Multi-Task Learning for Lung Nodule Segmentation and Diagnosis

GCS-ICHNet: Assessment of Intracerebral Hemorrhage Prognosis using Self-Attention with Domain Knowledge Integration

Automatic Segmentation of Intracranial Hemorrhage in Computed Tomography Scans with Convolution Neural Networks

Multi-scale Perception and Feature Refinement Network for multi-class segmentation of intracerebral hemorrhage in CT images

A Fully Automated Multimodal MRI-Based Multi-Task Learning for Glioma Segmentation and IDH Genotyping.

Multi-task Dictionary Learning based Convolutional Neural Network for Computer aided Diagnosis with Longitudinal Images

Multi-task Learning Approach for Intracranial Hemorrhage Prognosis

IHA-Net: An automatic segmentation framework for computer-tomography of tiny intracerebral hemorrhage based on improved attention U-net

Multi-task Deep Learning for Joint Tumor Segmentation and Outcome Prediction in Head and Neck Cancer.

Deep learning-based computed tomography image segmentation and volume measurement of intracerebral hemorrhage

Enhancing CT Image synthesis from multi-modal MRI data based on a multi-task neural network framework