Abstract:Infrared and visible image fusion aims to reconstruct fused images with comprehensive visual information by merging the complementary features of source images captured by different imaging sensors. This technology has been widely used in civil and military fields, such as urban security monitoring, remote sensing measurement, and battlefield reconnaissance. However, the existing methods still suffer from the preset fusion strategies that cannot be adjustable to different fusion demands and the loss of information during the feature propagation process, thereby leading to the poor generalization ability and limited fusion performance. Therefore, we propose an unsupervised end-to-end network with learnable fusion strategy for infrared and visible image fusion in this paper. The presented network mainly consists of three parts, including the feature extraction module, the fusion strategy module, and the image reconstruction module. First, in order to preserve more information during the process of feature propagation, dense connections and residual connections are applied to the feature extraction module and the image reconstruction module, respectively. Second, a new convolutional neural network is designed to adaptively learn the fusion strategy, which is able to enhance the generalization ability of our algorithm. Third, due to the lack of ground truth in fusion tasks, a loss function that consists of saliency loss and detail loss is exploited to guide the training direction and balance the retention of different types of information. Finally, the experimental results verify that the proposed algorithm delivers competitive performance when compared with several state-of-the-art algorithms in terms of both subjective and objective evaluations. Our codes are available at https://github.com/MinjieWan/Unsupervised-end-to-end-infrared-and-visible-image-fusion-network-using-learnable-fusion-strategy.

Target recognition with fusion of visible and infrared images based on mutual learning

Fusion of infrared and visual images through multiscale hybrid unidirectional total variation

Infrared and visible image fusion method based on saliency detection and target-enhancement

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

MIFFuse: A Multi-Level Feature Fusion Network for Infrared and Visible Images

Infrared and Visible Image Fusion Based on a Two-Stage Class Conditioned Auto-Encoder Network.

MFST: Multi-Modal Feature Self-Adaptive Transformer for Infrared and Visible Image Fusion

MAFusion: Multiscale Attention Network for Infrared and Visible Image Fusion

Multi-scale Convolutional Neural Networks and Saliency Weight Maps for Infrared and Visible Image Fusion

SMFD: an end-to-end infrared and visible image fusion model based on shared-individual multi-scale feature decomposition

Unsupervised end-to-end infrared and visible image fusion network using learnable fusion strategy

A Multi-Stage Visible and Infrared Image Fusion Network Based on Attention Mechanism

DCFusion: A Dual-Frequency Cross-Enhanced Fusion Network for Infrared and Visible Image Fusion.

Visible and Infrared Image Fusion Based on Attention and Multiscale Residuals

MSCS: Multi-stage Feature Learning with Channel-Spatial Attention Mechanism for Infrared and Visible Image Fusion

Infrared and visible image fusion based on infrared background suppression

BCMFIFuse: A Bilateral Cross-Modal Feature Interaction-Based Network for Infrared and Visible Image Fusion

Infrared-visible Image Fusion Using Accelerated Convergent Convolutional Dictionary Learning

Integrating Parallel Attention Mechanisms and Multi-Scale Features for Infrared and Visible Image Fusion

Fusion of Infrared and Visible Images based on Spatial-Channel Attentional Mechanism

Infrared and Visible Image Fusion with Convolutional Neural Networks.