Abstract:Infrared and visible image fusion can generate a fusion image with clear texture and prominent goals under extreme conditions. This capability is important for all-day climate detection and other tasks. However, most existing fusion methods for extracting features from infrared and visible images are based on convolutional neural networks (CNNs). These methods often fail to make full use of the salient objects and texture features in the raw image, leading to problems such as insufficient texture details and low contrast in the fused images. To this end, we propose an unsupervised end-to-end Fusion Decomposition Network (FDNet) for infrared and visible image fusion. Firstly, we construct a fusion network that extracts gradient and intensity information from raw images, using multi-scale layers, depthwise separable convolution, and improved convolution block attention module (I-CBAM). Secondly, as the FDNet network is based on the gradient and intensity information of the image for feature extraction, gradient and intensity loss are designed accordingly. Intensity loss adopts the improved Frobenius norm to adjust the weighing values between the fused image and the two raw to select more effective information. The gradient loss introduces an adaptive weight block that determines the optimized objective based on the richness of texture information at the pixel scale, ultimately guiding the fused image to generate more abundant texture information. Finally, we design a single and dual channel convolutional layer decomposition network, which keeps the decomposed image as possible with the input raw image, forcing the fused image to contain richer detail information. Compared with various other representative image fusion methods, our proposed method not only has good subjective vision, but also achieves advanced fusion performance in objective evaluation.

Fusion that matters: convolutional fusion networks for visual recognition

Fusion of Low-Illuminance Visible and Near-Infrared Images Based on Convolutional Neural Networks

On the Exploration of Convolutional Fusion Networks for Visual Recognition.

IFCNN: A General Image Fusion Framework Based on Convolutional Neural Network.

AMFF-net: Adaptive Multi-Modal Feature Fusion Network for Image Classification

CCAFusion: Cross-Modal Coordinate Attention Network for Infrared and Visible Image Fusion

CFNet: an Infrared and Visible Image Compression Fusion Network

TCCFusion: An Infrared and Visible Image Fusion Method based on Transformer and Cross Correlation

FDNet: An end-to-end fusion decomposition network for infrared and visible images

Multilevel Features Fusion In Deep Convolutional Neural Networks

DCFusion: A Dual-Frequency Cross-Enhanced Fusion Network for Infrared and Visible Image Fusion.

MIFFuse: A Multi-Level Feature Fusion Network for Infrared and Visible Images

Self-Fusion Convolutional Neural Networks.

A Late Fusion Approach for Harnessing Multi-Cnn Model High-Level Features

CMFuse: Cross-Modal Features Mixing Via Convolution and MLP for Infrared and Visible Image Fusion

Gated Fusion of Infrared and Visible Light Images Based on CNN

FusionGCN: Multi-focus image fusion using superpixel features generation GCN and pixel-level feature reconstruction CNN

FusionCNN: a Remote Sensing Image Fusion Algorithm Based on Deep Convolutional Neural Networks

DTFusion: Infrared and Visible Image Fusion Based on Dense Residual PConv-ConvNeXt and Texture-Contrast Compensation

Where Elegance Meets Precision: Towards a Compact, Automatic, and Flexible Framework for Multi-modality Image Fusion and Applications

CFNet: Context Fusion Network for Multi-Focus Images