Abstract:Deep learning has become a popular technique for infrared and visible image fusion (IVIF) due to its powerful feature representation abilities. Existing methods often employ the convolutional neural network (CNN), Transformer, or mixed CNN-Transformer to treat different modalities indiscriminately. However, we observe that these homogeneous fusion networks (i.e., same network for cross-modal) struggle to handle the specific characteristics of each modality, leading to fused images being contaminated with interference or lacking modality-specific information. To alleviate this issue, we propose a heterogeneous dual-branch multi-cascade fusion network based on CNN and Transformer, named DCTNet, aims independently to maintain the focus of respective modalities, such as radiation intensity of infrared images and texture details of visible images. In DCTNet, the CNN branch is specifically designed to exploit local features in visible images by utillizing stacked residual dense CNNs, while the Transformer branch is tailored to model the long-range dependencies, capturing the overall temperature distribution and thermal patterns of a scene in infrared images by cascading residual Transformers. In addition, we introduce an adaptive fusion interaction module (AFIM) that leverages attention mechanisms to adaptively highlight informative regions in the fused feature maps. The module assigns weights to these regions based on their contributions and dynamically merges features from the dual-branch at multiple levels. Experimental results on three public datasets demonstrate that the proposed method outperforms state-of-the-art methods in terms of quantitative and qualitative evaluations. Moreover, we showcase the promising performance of DCTNet in downstream object detection and semantic segmentation applications on a widely-accepted benchmark.

CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion

DCAFuse: Dual-Branch Diffusion-CNN Complementary Feature Aggregation Network for Multi-Modality Image Fusion

CMFuse: Cross-Modal Features Mixing Via Convolution and MLP for Infrared and Visible Image Fusion

Correlation-Guided Discriminative Cross-Modality Features Network for Infrared and Visible Image Fusion

MBHFuse: A Multi- Branch Heterogeneous Global and Local Infrared and Visible Image Fusion with Differential Convolutional Amplification Features

DPACFuse: Dual-Branch Progressive Learning for Infrared and Visible Image Fusion with Complementary Self-Attention and Convolution

DCFusion: Difference correlation-driven fusion mechanism of infrared and visible images

MDC-RHT: Multi-Modal Medical Image Fusion via Multi-Dimensional Dynamic Convolution and Residual Hybrid Transformer

DCFusion: Dual-Headed Fusion Strategy and Contextual Information Awareness for Infrared and Visible Remote Sensing Image

MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusion

DCMFNet: Deep Cross-Modal Fusion Network for Different Modalities with Iterative Gated Fusion

DAF-Net: A Dual-Branch Feature Decomposition Fusion Network with Domain Adaptive for Infrared and Visible Image Fusion

DCTNet: A Heterogeneous Dual-Branch Multi-Cascade Network for Infrared and Visible Image Fusion

DSAFuse: Infrared and Visible Image Fusion Via Dual-Branch Spatial Adaptive Feature Extraction

AFDFusion: an Adaptive Frequency Decoupling Fusion Network for Multi-Modality Image

CIRF: Coupled Image Reconstruction and Fusion Strategy for Deep Learning Based Multi-Modal Image Fusion

TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion

GLFuse: A Global and Local Four-Branch Feature Extraction Network for Infrared and Visible Image Fusion

BCMFIFuse: A Bilateral Cross-Modal Feature Interaction-Based Network for Infrared and Visible Image Fusion

DDRF: Dual-branch decomposition and reconstruction architecture for infrared and visible image fusion

DCFusion: A Dual-Frequency Cross-Enhanced Fusion Network for Infrared and Visible Image Fusion.