Abstract:Medical image fusion (MIF) has received painstaking attention due to its diverse medical applications in response to accurately diagnosing clinical images. Numerous MIF methods have been proposed to date, but the fused image suffers from poor contrast, non-uniform illumination, noise presence, and improper fusion strategies, resulting in an inadequate sparse representation of significant features. This paper proposes the morphological preprocessing method to address the non-uniform illumination and noise by the bottom-hat–top-hat strategy. Then, grey-principal component analysis (grey-PCA) is used to transform RGB images into gray images that can preserve detailed features. After that, the local shift-invariant shearlet transform (LSIST) method decomposes the images into the low-pass (LP) and high-pass (HP) sub-bands, efficiently restoring all significant characteristics in various scales and directions. The HP sub-bands are fed to two branches of the Siamese convolutional neural network (CNN) by process of feature detection, initial segmentation, and consistency verification to effectively capture smooth edges, and textures. While the LP sub-bands are fused by employing local energy fusion using the averaging and selection mode to restore the energy information. The proposed method is validated by subjective and objective quality assessments. The subjective evaluation is conducted by a user case study in which twelve field specialists verified the superiority of the proposed method based on precise details, image contrast, noise in the fused image, and no loss of information. The supremacy of the proposed method is further justified by obtaining 0.6836 to 0.8794, 0.5234 to 0.6710, and 3.8501 to 8.7937 gain for QFAB, CRR, and AG and noise reduction from 0.3397 to 0.1209 over other methods for objective parameters.

A Medical Image Fusion Method Based on Convolutional Neural Networks

Fusion of Low-Illuminance Visible and Near-Infrared Images Based on Convolutional Neural Networks

Multimodal medical image fusion based on interval gradients and convolutional neural networks

A Medical Image Fusion Method Based on SIFT and Deep Convolutional Neural Network in the SIST Domain

CT and MRI Medical Image Fusion Using Noise-Removal and Contrast Enhancement Scheme with Convolutional Neural Network

Multi-modal medical image fusion using improved dual-channel PCNN

Medical image fusion based on improved multi-scale morphology gradient-weighted local energy and visual saliency map

Medical image fusion with deep neural networks

Multimodal medical image fusion and classification using deep learning techniques

Multimodal MRI Volumetric Data Fusion With Convolutional Neural Networks

Medical Image Fusion Method Based on Coupled Neural P Systems in Nonsubsampled Shearlet Transform Domain

Multimodal Medical Supervised Image Fusion Method by CNN

Medical image fusion via discrete stationary wavelet transform and an enhanced radial basis function neural network

Medical Image Fusion with Parameter-Adaptive Pulse Coupled Neural Network in Nonsubsampled Shearlet Transform Domain

Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space

Infrared and Visible Image Fusion with Convolutional Neural Networks.

Adaptive spatial and frequency experts fusion network for medical image fusion

[CT and MRI fusion based on generative adversarial network and convolutional neural networks under image enhancement].

Multi-modal medical image fusion based on densely-connected high-resolution CNN and hybrid transformer

An ensemble deep learning model for medical image fusion with Siamese neural networks and VGG-19

Medical image fusion based on sparse representation and neighbor energy activity