Abstract:To obtain fused images with excellent contrast, distinct target edges, and well-preserved details, we propose an adaptive image fusion network called the adjacent feature shuffle-fusion network (AFSFusion). The proposed network adopts a UNet-like architecture and incorporates key refinements to enhance network architecture and loss functions. Regarding the network architecture, the proposed two-branch adjacent feature fusion module, called AFSF, expands the number of channels to fuse the feature channels of several adjacent convolutional layers in the first half of the AFSFusion, enhancing its ability to extract, transmit, and modulate feature information. We replace the original rectified linear unit (ReLU) with leaky ReLU to alleviate the problem of gradient disappearance and add a channel shuffling operation at the end of AFSF to facilitate information interaction capability between features. Concerning loss functions, we propose an adaptive weight adjustment (AWA) strategy to assign weight values to the corresponding pixels of the infrared (IR) and visible images in the fused images, according to the VGG16 gradient feature response of the IR and visible images. This strategy efficiently handles different scene contents. After normalization, the weight values are used as weighting coefficients for the two sets of images. The weighting coefficients are applied to three loss items simultaneously: mean square error (MSE), structural similarity (SSIM), and total variation (TV), resulting in clearer objects and richer texture detail in the fused images. We conducted a series of experiments on several benchmark databases, and the results demonstrate the effectiveness of the proposed network architecture and the superiority of the proposed network compared to other state-of-the-art fusion methods. It ranks first in several objective metrics, showing the best performance and exhibiting sharper and richer edges of specific targets, which is more in line with human visual perception. The remarkable enhancement in performance is ascribed to the proposed AFSF module and AWA strategy, enabling balanced feature extraction, fusion, and modulation of image features throughout the process.

Unsupervised end-to-end infrared and visible image fusion network using learnable fusion strategy

Fusion of Low-Illuminance Visible and Near-Infrared Images Based on Convolutional Neural Networks

IVJDN: an End-to-End Network for Joint Infrared and Visible Image Fusion and Detection

Infrared and Visible Image Fusion Based on a Two-Stage Class Conditioned Auto-Encoder Network.

VIF-Net: an Unsupervised Framework for Infrared and Visible Image Fusion

Infrared and visible image fusion based on residual dense network and gradient loss

Infrared and Visible Image Fusion with Supervised Convolutional Neural Network

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

Infrared and visible image fusion with entropy-based adaptive fusion module and mask-guided convolutional neural network

Weakly Supervised Infrared and Visible Image Fusion Based on Attention Mechanism

EV-Fusion: A Novel Infrared and Low-Light Color Visible Image Fusion Network Integrating Unsupervised Visible Image Enhancement

STFNet: Self-Supervised Transformer for Infrared and Visible Image Fusion

Multi-Modality Image Fusion and Object Detection Based on Semantic Information

A Multi-Stage Visible and Infrared Image Fusion Network Based on Attention Mechanism

Infrared and Visible Image Fusion with Convolutional Neural Networks.

Visible and Infrared Image Fusion Based on Attention and Multiscale Residuals

AFSFusion: an Adjacent Feature Shuffle Combination Network for Infrared and Visible Image Fusion

A Semantic-Aware and Multi-Guided Network for Infrared-Visible Image Fusion

MLFFusion: Multi-level Feature Fusion Network with Region Illumination Retention for Infrared and Visible Image Fusion

An Infrared and Visible Image Fusion Framework Based on Dual Scale Decomposition and Learnable Attention Fusion Strategy

A Self-Supervised Method for Infrared and Visible Image Fusion