Abstract:Medical imaging is indispensable for accurate diagnosis and effective treatment, with modalities like MRI and CT providing diverse yet complementary information. Traditional image fusion methods, while essential in consolidating information from multiple modalities, often suffer from poor image quality and loss of crucial details due to inadequate handling of semantic information and limited feature extraction capabilities. This paper introduces a novel medical image fusion technique leveraging unsupervised image segmentation to enhance the semantic understanding of the fusion process. The proposed method, named DUSMIF, employs a multi-branch, multi-scale deep learning architecture that integrates advanced attention mechanisms to refine the feature extraction and fusion processes. An innovative approach that utilizes unsupervised image segmentation to extract semantic information is introduced, which is then integrated into the fusion process. This not only enhances the semantic relevance of the fused images but also improves the overall fusion quality. The paper proposes a sophisticated network structure that extracts and fuses features at multiple scales and across multiple branches. This structure is designed to capture a comprehensive range of image details and contextual information, significantly improving the fusion outcomes. Multiple attention mechanisms are incorporated to selectively emphasize important features and integrate them effectively across different modalities and scales. This approach ensures that the fused images maintain high quality and detail fidelity. A joint loss function combining content loss, structural similarity loss, and semantic loss is formulated. This function not only guides the network in preserving image brightness and texture but also ensures that the fused image closely resembles the source images in both content and structure. The proposed method demonstrates superior performance over existing fusion techniques in objective assessments and subjective evaluations, confirming its effectiveness in enhancing the diagnostic utility of fused medical images.

Multi-level multi-type self-generated knowledge fusion for cardiac ultrasound segmentation

Neural Multi-Atlas Label Fusion: Application to Cardiac MR Images

MSMHSA-DeepLab V3+: An Effective Multi-Scale, Multi-Head Self-Attention Network for Dual-Modality Cardiac Medical Image Segmentation

Label-free segmentation from cardiac ultrasound using self-supervised learning

Automated cardiac segmentation of cross-modal medical images using unsupervised multi-domain adaptation and spatial neural attention structure

Causal knowledge fusion for 3D cross-modality cardiac image segmentation

Multi-Scale Feature Fusion Convolutional Neural Network for Concurrent Segmentation of Left Ventricle and Myocardium in Cardiac MR Images

Deep Fusion Net for Multi-atlas Segmentation: Application to Cardiac MR Images

EchoDFKD: Data-Free Knowledge Distillation for Cardiac Ultrasound Segmentation using Synthetic Data

Medical image segmentation based on self-supervised hybrid fusion network

A multibranch and multiscale neural network based on semantic perception for multimodal medical image fusion

Regional perception and multi-scale feature fusion network for cardiac segmentation

Medical Image Segmentation Based on Multi-Modal Convolutional Neural Network: Study on Image Fusion Schemes

Unsupervised Cardiac Segmentation Utilizing Synthesized Images from Anatomical Labels

Deep Learning-Based Image Segmentation on Multimodal Medical Imaging

Semi-supervised segmentation of echocardiography videos via noise-resilient spatiotemporal semantic calibration and fusion

Self-supervised multi-modal fusion network for multi-modal thyroid ultrasound image diagnosis

Multi-source adversarial transfer learning for ultrasound image segmentation with limited similarity

Simultaneous Segmentation of Fetal Hearts and Lungs for Medical Ultrasound Images Via an Efficient Multi-scale Model Integrated with Attention Mechanism

GL-Fusion: Global-Local Fusion Network for Multi-view Echocardiogram Video Segmentation

Enhancing Cardiac MRI Segmentation via Classifier-Guided Two-Stage Network and All-Slice Information Fusion Transformer