Abstract:Infrared and visible image fusion can generate a fusion image with clear texture and prominent goals under extreme conditions. This capability is important for all-day climate detection and other tasks. However, most existing fusion methods for extracting features from infrared and visible images are based on convolutional neural networks (CNNs). These methods often fail to make full use of the salient objects and texture features in the raw image, leading to problems such as insufficient texture details and low contrast in the fused images. To this end, we propose an unsupervised end-to-end Fusion Decomposition Network (FDNet) for infrared and visible image fusion. Firstly, we construct a fusion network that extracts gradient and intensity information from raw images, using multi-scale layers, depthwise separable convolution, and improved convolution block attention module (I-CBAM). Secondly, as the FDNet network is based on the gradient and intensity information of the image for feature extraction, gradient and intensity loss are designed accordingly. Intensity loss adopts the improved Frobenius norm to adjust the weighing values between the fused image and the two raw to select more effective information. The gradient loss introduces an adaptive weight block that determines the optimized objective based on the richness of texture information at the pixel scale, ultimately guiding the fused image to generate more abundant texture information. Finally, we design a single and dual channel convolutional layer decomposition network, which keeps the decomposed image as possible with the input raw image, forcing the fused image to contain richer detail information. Compared with various other representative image fusion methods, our proposed method not only has good subjective vision, but also achieves advanced fusion performance in objective evaluation.

Infrared and visible image fusion based on double fluid pyramids and multi-scale gradient residual block

Infrared and visible image fusion based on residual dense network and gradient loss

A Full-Scale Hierarchical Encoder-Decoder Network with Cascading Edge-Prior for Infrared and Visible Image Fusion

Visible and Infrared Image Fusion Based on Attention and Multiscale Residuals

A Visible and Infrared Image Fusion Framework Based on Dual-Path Encoder-Decoder and Multi-Scale Discrete Wavelet Transform

FDNet: An end-to-end fusion decomposition network for infrared and visible images

Image Fusion Using a Multi-Level Image Decomposition and Fusion Method.

DGFusion: an Effective Dynamic Generalizable Network for Infrared and Visible Image Fusion

Infrared and Visible Image Fusion Based on Filtering Enhancement

Infrared and visible image fusion based on visibility enhancement and hybrid multiscale decomposition

An Infrared and Visible Image Fusion Framework Based on Dual Scale Decomposition and Learnable Attention Fusion Strategy

Laplacian Pyramid Fusion Network with Hierarchical Guidance for Infrared and Visible Image Fusion

Image Fusion Using Y-Net-based Extractor and Global-Local Discriminator

SGFusion: A Saliency Guided Deep-Learning Framework for Pixel-Level Image Fusion

An Information Retention and Feature Transmission Network for Infrared and Visible Image Fusion

Infrared and visible image fusion with entropy-based adaptive fusion module and mask-guided convolutional neural network

A Dual-branch Network for Infrared and Visible Image Fusion

MGFuse: An Infrared and Visible Image Fusion Algorithm Based on Multiscale Decomposition Optimization and Gradient-Weighted Local Energy

DSG-Fusion: Infrared and visible image fusion via generative adversarial networks and guided filter

SFPFusion: An Improved Vision Transformer Combining Super Feature Attention and Wavelet-Guided Pooling for Infrared and Visible Images Fusion

An end-to-end multi-scale network based on autoencoder for infrared and visible image fusion