Abstract:Medical image fusion plays a vital role in clinical applications by combining complementary information from different imaging modalities, such as MRI, CT, PET, and SPECT, into a single, more informative image. This process is essential for noninvasive diagnostics, treatment planning, and image-guided surgeries. Despite the critical need for effective image fusion, current methods, particularly those based on pre-trained models, often face limitations. These include insufficient training on medical image datasets, leading to suboptimal feature representation, and the degradation of important image attributes such as brightness and contrast, which can diminish the diagnostic value of the fused images. To address these challenges, this paper proposes a novel medical image fusion method that combines the strengths of Bilateral Texture Filtering (BTF) and transfer learning with a modified ResNet-101 network (M_ResNet-101). The method begins with the application of BTF to decompose the input images into texture and detail layers, preserving structural integrity while effectively separating relevant features. Subsequently, the M_ResNet-101 network, trained specifically on a diverse medical image dataset, is employed to classify and fuse the detail layers of MRI, CT, PET, and SPECT images, ensuring that the most critical diagnostic information is integrated accurately. Furthermore, to address the common issue of brightness and contrast degradation in the fusion process, we apply the Crayfish Optimization Algorithm (COA). This algorithm adaptively controls the fusion of texture layers, optimizing the output to maintain the inherent qualities of the original images. The COA ensures that the fused image retains high visual quality, which is crucial for effective diagnosis and analysis. The results demonstrate that our proposed method significantly outperforms existing state-of-the-art techniques, achieving superior performance in both objective metrics and subjective assessments. The fused images produced by our method exhibit enhanced clarity, improved contrast, and better preservation of fine details.

Medical Image Fusion Using Bilateral Texture Filtering

Medical image fusion based on quasi-cross bilateral filtering

Multi-Sensor Medical-Image Fusion Technique Based on Embedding Bilateral Filter in Least Squares and Salient Detection

Multi-Modal Medical Image Fusion Via Three-Scale Decomposition and Structure Similarity Balanced Optimization

Medical image fusion based on improved multi-scale morphology gradient-weighted local energy and visual saliency map

Medical Image Fusion Algorithm Based on Nonlinear Approximation of Contourlet Transform and Regional Features.

A Three-Layer Decomposition Method Based on Structural Texture Perception for Fusion of CT and MRI Images

A Fusion Algorithm for Medical Structural and Functional Images Based on Adaptive Image Decomposition

A Medical Image Fusion Method Based on Detail Enhancement Decomposition.

Regions Preserving Edge Enhancement For Multisensor-Based Medical Image Fusion

Image Fusion with Structural Saliency Measure and Content Adaptive Consistency Verification.

Adaptive Decomposition Method for Multi-Modal Medical Image Fusion.

An Algorithm of Medical Image Fusion Based on Shearlet Transformation

Tri-Modal Medical Image Fusion Based on Cartoon-Texture Decomposition and Sparse Representation

MIF-BTF-MRN: Medical image fusion based on the bilateral texture filter and transfer learning with the ResNet-101 network

Medical Image Fusion Based on Pixel-Level Nonlocal Self-similarity Prior and Optimization.

Multimodal Medical Image Fusion Based on Joint Bilateral Filter and Local Gradient Energy.

Medical image fusion based on sparse representation and neighbor energy activity

Tri-modal Medical Image Fusion Based on Adaptive Energy Choosing Scheme and Sparse Representation

C2IENet: Multi-branch medical image fusion based on contrastive constraint features and information exchange

An Infrared and Visible Image Fusion Algorithm Method Based on a Dual Bilateral Least Squares Hybrid Filter