Abstract:Medical image fusion plays a vital role in clinical applications by combining complementary information from different imaging modalities, such as MRI, CT, PET, and SPECT, into a single, more informative image. This process is essential for noninvasive diagnostics, treatment planning, and image-guided surgeries. Despite the critical need for effective image fusion, current methods, particularly those based on pre-trained models, often face limitations. These include insufficient training on medical image datasets, leading to suboptimal feature representation, and the degradation of important image attributes such as brightness and contrast, which can diminish the diagnostic value of the fused images. To address these challenges, this paper proposes a novel medical image fusion method that combines the strengths of Bilateral Texture Filtering (BTF) and transfer learning with a modified ResNet-101 network (M_ResNet-101). The method begins with the application of BTF to decompose the input images into texture and detail layers, preserving structural integrity while effectively separating relevant features. Subsequently, the M_ResNet-101 network, trained specifically on a diverse medical image dataset, is employed to classify and fuse the detail layers of MRI, CT, PET, and SPECT images, ensuring that the most critical diagnostic information is integrated accurately. Furthermore, to address the common issue of brightness and contrast degradation in the fusion process, we apply the Crayfish Optimization Algorithm (COA). This algorithm adaptively controls the fusion of texture layers, optimizing the output to maintain the inherent qualities of the original images. The COA ensures that the fused image retains high visual quality, which is crucial for effective diagnosis and analysis. The results demonstrate that our proposed method significantly outperforms existing state-of-the-art techniques, achieving superior performance in both objective metrics and subjective assessments. The fused images produced by our method exhibit enhanced clarity, improved contrast, and better preservation of fine details.

A novel approach using the local energy function and its variations for medical image fusion

A novel approach using structure tensor for medical image fusion

Medical image fusion based on enhanced three-layer image decomposition and Chameleon swarm algorithm

Combining spectral total variation with dynamic threshold neural P systems for medical image fusion

Multi-modal medical image fusion based on equilibrium optimizer algorithm and local energy functions

A new approach to medical image fusion based on the improved Extended difference-of-Gaussians combined with the Coati optimization algorithm

Medical image fusion based on improved multi-scale morphology gradient-weighted local energy and visual saliency map

Multiresolution approach on medical image fusion by modified local energy

Medical image fusion based on sparse representation and neighbor energy activity

A new medical image enhancement algorithm using adaptive parameters

Medical Image Fusion Based on Pixel-Level Nonlocal Self-similarity Prior and Optimization.

Medical image fusion using multi-level local extrema

Medical Image Fusion Algorithm Based on Nonlinear Approximation of Contourlet Transform and Regional Features.

A Medical Image Fusion Method Based on Detail Enhancement Decomposition.

Medical image fusion via discrete stationary wavelet transform and an enhanced radial basis function neural network

Brain Medical Image Fusion Using L2-Norm-Based Features and Fuzzy-Weighted Measurements in 2-D Littlewood–Paley EWT Domain

A Novel Wavelet Medical Image Fusion Method

Multimodal Medical Image Fusion Based on Joint Bilateral Filter and Local Gradient Energy.

Medical Image Fusion Via PCNN Based on Edge Preservation and Improved Sparse Representation in NSST Domain

A Medical Image Fusion Method Based on Convolutional Neural Networks

MIF-BTF-MRN: Medical image fusion based on the bilateral texture filter and transfer learning with the ResNet-101 network