Abstract:ABSTRACT Throughout the past 20 years, medical imaging has found extensive application in clinical diagnosis. Doctors may find it difficult to diagnose diseases using only one imaging modality. The main objective of multimodal medical image fusion (MMIF) is to improve both the accuracy and quality of clinical assessments by extracting structural and spectral information from source images. This study proposes a novel MMIF method to assist doctors and postoperations such as image segmentation, classification, and further surgical procedures. Initially, the intensity‐hue‐saturation (IHS) model is utilized to decompose the positron emission tomography (PET)/single photon emission computed tomography (SPECT) image, followed by a hue‐angle mapping method to discriminate high‐ and low‐activity regions in the PET images. Then, a proposed structure feature adjustment (SFA) mechanism is used as a fusion strategy for high‐ and low‐activity regions to obtain structural and anatomical details with minimum color distortion. In the second step, a new multi‐discriminator generative adversarial network (MDcGAN) approach is proposed for obtaining the final fused image. The qualitative and quantitative results demonstrate that the proposed method is superior to existing MMIF methods in preserving the structural, anatomical, and functional details of the PET/SPECT images. Through our assessment, involving visual analysis and subsequent verification using statistical metrics, it becomes evident that color changes contribute substantial visual information to the fusion of PET and MR images. The quantitative outcomes demonstrate that, in the majority of cases, the proposed algorithm consistently outperformed other methods. Yet, in a few instances, it achieved the second‐highest results. The validity of the proposed method was confirmed using diverse modalities, encompassing a total of 1012 image pairs.

MRI and PET/SPECT fusion with pseudo spatial frequency in local laplacian filter domain

Image Fusion of CT and MR with Sparse Representation in NSST Domain

MRI and PET image fusion using the nonparametric density model and the theory of variable-weight

Multi-modality Anatomical and Functional Medical Image Fusion Based on Simplified-spatial Frequency-pulse Coupled Neural Networks and Region Energy-weighted Average Strategy in Non-sub Sampled Contourlet Transform Domain.

High-Quality Fusion and Visualization for MR-PET Brain Tumor Images via Multi-Dimensional Features

Medical image fusion based on sparse representation and neighbor energy activity

A fusion algorithm based on composite decomposition for PET and MRI medical images

Medical image fusion based on improved multi-scale morphology gradient-weighted local energy and visual saliency map

Medical Image Fusion with Parameter-Adaptive Pulse Coupled Neural Network in Nonsubsampled Shearlet Transform Domain

Multimodal medical image fusion based on IHS and PCA

Adaptive spatial and frequency experts fusion network for medical image fusion

Multimodal medical image fusion based on visual saliency map and multichannel dynamic threshold neural P systems in sub-window variance filter domain

NSCT-DCT based Fourier Analysis for Fusion of Multimodal Images

MRI/CT Fusion Based on Latent Low Rank Representation and Gradient Transfer

CT and MRI Medical Image Fusion Using Noise-Removal and Contrast Enhancement Scheme with Convolutional Neural Network

Performance of Medical Image Fusion in High-level Analysis Tasks: A Mutual Enhancement Framework for Unaligned PAT and MRI Image Fusion

HID: The Hybrid Image Decomposition Model for MRI and CT Fusion

CT and MRI Image Fusion Based on Multiscale Decomposition Method and Hybrid Approach.

Multimodal medical image fusion with convolution sparse representation and mutual information correlation in NSST domain

Medical Image Fusion for Multiple Diseases Features Enhancement

A Medical Image Fusion Method Based on Convolutional Neural Networks