Abstract:ABSTRACT Throughout the past 20 years, medical imaging has found extensive application in clinical diagnosis. Doctors may find it difficult to diagnose diseases using only one imaging modality. The main objective of multimodal medical image fusion (MMIF) is to improve both the accuracy and quality of clinical assessments by extracting structural and spectral information from source images. This study proposes a novel MMIF method to assist doctors and postoperations such as image segmentation, classification, and further surgical procedures. Initially, the intensity‐hue‐saturation (IHS) model is utilized to decompose the positron emission tomography (PET)/single photon emission computed tomography (SPECT) image, followed by a hue‐angle mapping method to discriminate high‐ and low‐activity regions in the PET images. Then, a proposed structure feature adjustment (SFA) mechanism is used as a fusion strategy for high‐ and low‐activity regions to obtain structural and anatomical details with minimum color distortion. In the second step, a new multi‐discriminator generative adversarial network (MDcGAN) approach is proposed for obtaining the final fused image. The qualitative and quantitative results demonstrate that the proposed method is superior to existing MMIF methods in preserving the structural, anatomical, and functional details of the PET/SPECT images. Through our assessment, involving visual analysis and subsequent verification using statistical metrics, it becomes evident that color changes contribute substantial visual information to the fusion of PET and MR images. The quantitative outcomes demonstrate that, in the majority of cases, the proposed algorithm consistently outperformed other methods. Yet, in a few instances, it achieved the second‐highest results. The validity of the proposed method was confirmed using diverse modalities, encompassing a total of 1012 image pairs.

Multimodal Fusion Learning with Dual Attention for Medical Imaging

Multimodal Information Fusion for Glaucoma and DR Classification

Multimodal medical image fusion and classification using deep learning techniques

Adaptive Multimodal Image Fusion with a Deep Pyramidal Residual Learning Network

Interpretation on Deep Multimodal Fusion for Diagnostic Classification

Fusion of medical imaging and electronic health records with attention and multi-head machanisms

Medical Image Fusion for Multiple Diseases Features Enhancement

Multimodal Medical Imaging Using Modern Deep Learning Approaches

Two-Dimensional Attentive Fusion for Multi-Modal Learning of Neuroimaging and Genomics Data

A review of deep learning-based information fusion techniques for multimodal medical image classification

Feature extraction of multimodal medical image fusion using novel deep learning and contrast enhancement method

Edge-Enhanced Dilated Residual Attention Network for Multimodal Medical Image Fusion

Multimodal Densenet

Three-Dimensional Medical Image Fusion with Deformable Cross-Attention

Multi-modal Medical Data Fusion using Deep Learning

Multimodal medical image fusion algorithm in the era of big data

Multimodal learning for fetal distress diagnosis using a multimodal medical information fusion framework

MDA: An Interpretable and Scalable Multi-Modal Fusion under Missing Modalities and Intrinsic Noise Conditions

Missing-modality Enabled Multi-modal Fusion Architecture for Medical Data

Multimodal MRI Volumetric Data Fusion With Convolutional Neural Networks

A New Multimodal Medical Image Fusion based on Laplacian Autoencoder with Channel Attention