Abstract:Infrared and visible image fusion aims to reconstruct fused images with comprehensive visual information by merging the complementary features of source images captured by different imaging sensors. This technology has been widely used in civil and military fields, such as urban security monitoring, remote sensing measurement, and battlefield reconnaissance. However, the existing methods still suffer from the preset fusion strategies that cannot be adjustable to different fusion demands and the loss of information during the feature propagation process, thereby leading to the poor generalization ability and limited fusion performance. Therefore, we propose an unsupervised end-to-end network with learnable fusion strategy for infrared and visible image fusion in this paper. The presented network mainly consists of three parts, including the feature extraction module, the fusion strategy module, and the image reconstruction module. First, in order to preserve more information during the process of feature propagation, dense connections and residual connections are applied to the feature extraction module and the image reconstruction module, respectively. Second, a new convolutional neural network is designed to adaptively learn the fusion strategy, which is able to enhance the generalization ability of our algorithm. Third, due to the lack of ground truth in fusion tasks, a loss function that consists of saliency loss and detail loss is exploited to guide the training direction and balance the retention of different types of information. Finally, the experimental results verify that the proposed algorithm delivers competitive performance when compared with several state-of-the-art algorithms in terms of both subjective and objective evaluations. Our codes are available at https://github.com/MinjieWan/Unsupervised-end-to-end-infrared-and-visible-image-fusion-network-using-learnable-fusion-strategy.

An end-to-end multi-scale network based on autoencoder for infrared and visible image fusion

Fusion of Low-Illuminance Visible and Near-Infrared Images Based on Convolutional Neural Networks

Infrared and Visible Image Fusion Based on a Two-Stage Class Conditioned Auto-Encoder Network.

An infrared and visible image fusion network based on multi‐scale feature cascades and non‐local attention

Visible and Infrared Image Fusion Based on Attention and Multiscale Residuals

Advancing infrared and visible image fusion with an enhanced multiscale encoder and attention-based networks

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

When Image Decomposition Meets Deep Learning: A Novel Infrared and Visible Image Fusion Method

Interactive residual coordinate attention and contrastive learning for infrared and visible image fusion in triple frequency bands

Multi-scale attention-based lightweight network with dilated convolutions for infrared and visible image fusion

Infrared and Visible Image Fusion with Convolutional Neural Networks.

A joint convolution auto-encoder network for infrared and visible image fusion

Unsupervised end-to-end infrared and visible image fusion network using learnable fusion strategy

Infrared-visible Image Fusion Based on Regional Attention Auto-Encoder

Fusion of Infrared and Visible Images Based on Three-Scale Decomposition and ResNet Feature Transfer

Infrared and visible image fusion with entropy-based adaptive fusion module and mask-guided convolutional neural network

A Semantic-Aware and Multi-Guided Network for Infrared-Visible Image Fusion

Infrared and visible image fusion based on double fluid pyramids and multi-scale gradient residual block

IR-MSDNet: Infrared and Visible Image Fusion Based On Infrared Features and Multiscale Dense Network

Infrared and Visible Image Fusion Based on Adversarial Feature Extraction and Stable Image Reconstruction

MIFFuse: A Multi-Level Feature Fusion Network for Infrared and Visible Images