Abstract:Infrared and visible image fusion aims to reconstruct fused images with comprehensive visual information by merging the complementary features of source images captured by different imaging sensors. This technology has been widely used in civil and military fields, such as urban security monitoring, remote sensing measurement, and battlefield reconnaissance. However, the existing methods still suffer from the preset fusion strategies that cannot be adjustable to different fusion demands and the loss of information during the feature propagation process, thereby leading to the poor generalization ability and limited fusion performance. Therefore, we propose an unsupervised end-to-end network with learnable fusion strategy for infrared and visible image fusion in this paper. The presented network mainly consists of three parts, including the feature extraction module, the fusion strategy module, and the image reconstruction module. First, in order to preserve more information during the process of feature propagation, dense connections and residual connections are applied to the feature extraction module and the image reconstruction module, respectively. Second, a new convolutional neural network is designed to adaptively learn the fusion strategy, which is able to enhance the generalization ability of our algorithm. Third, due to the lack of ground truth in fusion tasks, a loss function that consists of saliency loss and detail loss is exploited to guide the training direction and balance the retention of different types of information. Finally, the experimental results verify that the proposed algorithm delivers competitive performance when compared with several state-of-the-art algorithms in terms of both subjective and objective evaluations. Our codes are available at https://github.com/MinjieWan/Unsupervised-end-to-end-infrared-and-visible-image-fusion-network-using-learnable-fusion-strategy.

IVJDN: an End-to-End Network for Joint Infrared and Visible Image Fusion and Detection

IAIFNet: An Illumination-Aware Infrared and Visible Image Fusion Network

An Interactively Reinforced Paradigm for Joint Infrared-Visible Image Fusion and Saliency Object Detection

Unsupervised end-to-end infrared and visible image fusion network using learnable fusion strategy

Low-light Infrared and Visible Image Fusion with Imbalanced Thermal Radiation Distribution

Infrared and Visible Image Fusion Based on Variational Auto-Encoder and Infrared Feature Compensation

Visible and Infrared Image Fusion Based on Attention and Multiscale Residuals

MDAN: Multilevel dual-branch attention network for infrared and visible image fusion

IR-MSDNet: Infrared and Visible Image Fusion Based On Infrared Features and Multiscale Dense Network

Infrared and Visible Image Fusion with Convolutional Neural Networks.

MIFFuse: A Multi-Level Feature Fusion Network for Infrared and Visible Images

Adaptive low light visual enhancement and high-significant target detection for infrared and visible image fusion

Infrared and visible image fusion with entropy-based adaptive fusion module and mask-guided convolutional neural network

EV-Fusion: A Novel Infrared and Low-Light Color Visible Image Fusion Network Integrating Unsupervised Visible Image Enhancement

Distillation-fusion-semantic unified driven network for infrared and visible image fusion

DDFNet-A: Attention-Based Dual-Branch Feature Decomposition Fusion Network for Infrared and Visible Image Fusion

Integrating Parallel Attention Mechanisms and Multi-Scale Features for Infrared and Visible Image Fusion

RAN: Infrared and Visible Image Fusion Network Based on Residual Attention Decomposition

Advancing infrared and visible image fusion with an enhanced multiscale encoder and attention-based networks

DCFusion: A Dual-Frequency Cross-Enhanced Fusion Network for Infrared and Visible Image Fusion.