Abstract:Glioma is a kind of brain disease with high incidence, high recurrence rate, high mortality, and low cure rate. To obtain accurate diagnosis results of brain glioma, doctors need to manually compare the imaging results of different modalities many times, which will increase the diagnosis time and reduce the diagnostic efficiency. Image fusion technology has been widely used in recent years to obtain information on multimodal medical images. This paper proposes a novel image fusion framework, target information enhanced image fusion network (TIEF), using cross-modal learning and information enhancement techniques. The framework consists of a multi-sequence feature extraction block, a feature selection block, and a fusion block. The multi-sequence feature extraction block consists of multiple sobel dense conv leaky ReLu block (SDCL-block). SDCL-block mainly realizes the extraction of edge features, shallow features, and deep features. The feature selection block identifies the feature channels with rich texture information and strong discrimination ability through the effective combination of global information entropy criterion and feature jump connection. The feature fusion block mainly comprises multi-head and spatial attention mechanisms, which can realize the fusion of intra-modality and inter-modality features. On this basis, considering the influence of tumor spatial location and structure information on the fusion results, a loss function is designed, which is a weighted combination of texture loss, structure loss, and saliency loss so that texture information from multimodal magnetic resonance imaging (MMRI) and saliency information from different anatomical structures of the brain can be fused at the same time to improve the expression ability of features. In this paper, the TIEF algorithm is trained and validated on the MMRI and (Single-Photon Emission Computed Tomography-MRI) SPECT-MRI datasets of glioma and generalized on the (Computed Tomography-MRI) CT-MRI dataset of meningioma to verify the performance of the TIEF algorithm. In the image fusion task, quantitative results showed that TIEF exhibited optimal or suboptimal performance in information entropy, spatial frequency, and average gradient metrics. Qualitative results indicate that the fused images can highlight tumor and edematous features. A downstream image segmentation task was used for evaluation to further verify TIEF's effectiveness. TIEF achieved the best results in both (Dice similarity coefficient) Dice and (Hausdorff distance 95%) HD95 segmentation metrics. In the generalization task, quantitative results indicated that TIEF obtained more information in the meningioma dataset. In conclusion, TIEF can effectively achieve cross-domain information acquisition and fusion and has robustness and generalization ability.

Efficient Utilization of Image Fusion and Interpolation for Medical Image Diagnosis applications

Utilization of image interpolation and fusion in brain tumor segmentation

An improved hybrid framework for brain tumor detection

Improving Brain MRI Image Segmentation Quality: A Hybrid Technique for Intensity Inhomogeneity Correction

Optimized multimodal medical image fusion framework using multi-scale geometric and multi-resolution geometric analysis

DWT-PCA Image Fusion Technique to Improve Segmentation Accuracy in Brain Tumor Analysis

An efficient wavelet-based image fusion for brain tumor detection and segmentation over PET and MRI image

Discrete Wavelet Transformation based Multimodal Medical Image Fusion for Disease Identification

A Framework of Fuzzy Information Fusion for the Segmentation of Brain Tumor Tissues on MR Images

An Efficient Framework for the Segmentation of Glioma Brain Tumor Using Image Fusion and Co-Active Adaptive Neuro Fuzzy Inference System Classification Method

Effective image fusion strategies in scientific signal processing disciplines: Application to cancer and carcinoma treatment planning

An efficient computerized decision support system for the analysis and 3D visualization of brain tumor

An Efficient Dwt And Intuitionistic Fuzzy Based Multimodality Medical Image Fusion

Effective Image Fusion of PET-MRI Brain Images Using Wavelet Transforms

Optimized Brain Tumor Detection: A Dual-Module Approach for MRI Image Enhancement and Tumor Classification

E-fuzzy feature fusion and thresholding for morphology segmentation of brain MRI modalities

Kernel Feature Selection to Fuse Multi-Spectral Mri Images for Brain Tumor Segmentation

Multimodal Medical Image Fusion Network Based on Target Information Enhancement

Brain Tumor Detection by Wavelet Based Fusion Technique

Investigating the Role of Image Fusion in Brain Tumor Classification Models Based on Machine Learning Algorithm for Personalized Medicine

Medical image segmentation approach based on hybrid adaptive differential evolution and crayfish optimizer