Abstract:Medical image fusion plays a vital role in clinical applications by combining complementary information from different imaging modalities, such as MRI, CT, PET, and SPECT, into a single, more informative image. This process is essential for noninvasive diagnostics, treatment planning, and image-guided surgeries. Despite the critical need for effective image fusion, current methods, particularly those based on pre-trained models, often face limitations. These include insufficient training on medical image datasets, leading to suboptimal feature representation, and the degradation of important image attributes such as brightness and contrast, which can diminish the diagnostic value of the fused images. To address these challenges, this paper proposes a novel medical image fusion method that combines the strengths of Bilateral Texture Filtering (BTF) and transfer learning with a modified ResNet-101 network (M_ResNet-101). The method begins with the application of BTF to decompose the input images into texture and detail layers, preserving structural integrity while effectively separating relevant features. Subsequently, the M_ResNet-101 network, trained specifically on a diverse medical image dataset, is employed to classify and fuse the detail layers of MRI, CT, PET, and SPECT images, ensuring that the most critical diagnostic information is integrated accurately. Furthermore, to address the common issue of brightness and contrast degradation in the fusion process, we apply the Crayfish Optimization Algorithm (COA). This algorithm adaptively controls the fusion of texture layers, optimizing the output to maintain the inherent qualities of the original images. The COA ensures that the fused image retains high visual quality, which is crucial for effective diagnosis and analysis. The results demonstrate that our proposed method significantly outperforms existing state-of-the-art techniques, achieving superior performance in both objective metrics and subjective assessments. The fused images produced by our method exhibit enhanced clarity, improved contrast, and better preservation of fine details.

BAM: Bilateral Activation Mechanism for Image Fusion

Aggregating Hierarchical Binary Activations for Image Retrieval

BAM: Bottleneck Attention Module

SBIT-Fuse: Infrared and visible image fusion based on Symmetrical Bilateral interaction and Transformer

MIF-BTF-MRN: Medical image fusion based on the bilateral texture filter and transfer learning with the ResNet-101 network

Why mamba is effective? Exploit Linear Transformer-Mamba Network for Multi-Modality Image Fusion

BCMFIFuse: A Bilateral Cross-Modal Feature Interaction-Based Network for Infrared and Visible Image Fusion

FusionMamba: Dynamic Feature Enhancement for Multimodal Image Fusion with Mamba

Improving Image Super-Resolution Via Feature Re-Balancing Fusion

MBRARN: multibranch residual attention reconstruction network for medical image fusion

FusionMamba: Efficient Remote Sensing Image Fusion with State Space Model

FusionMamba: Efficient Image Fusion with State Space Model

Cascaded Bilinear Mapping Collaborative Hybrid Attention Modality Fusion Model.

Blur Regional Features Based Infrared and Visible Image Fusion Using an Improved C3Net Model

Multi-Scale Bilateral Attention Fusion Network for Pansharpening

MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusion

A Novel State Space Model with Local Enhancement and State Sharing for Image Fusion

ReCoNet: Recurrent Correction Network for Fast and Efficient Multi-modality Image Fusion.

BTSFusion: Fusion of Infrared and Visible Image Via a Mechanism of Balancing Texture and Salience

ResCCFusion: Infrared and Visible Image Fusion Network Based on ResCC Module and Spatial Criss-Cross Attention Models

DDRF: Dual-branch decomposition and reconstruction architecture for infrared and visible image fusion