Abstract:To obtain fused images with excellent contrast, distinct target edges, and well-preserved details, we propose an adaptive image fusion network called the adjacent feature shuffle-fusion network (AFSFusion). The proposed network adopts a UNet-like architecture and incorporates key refinements to enhance network architecture and loss functions. Regarding the network architecture, the proposed two-branch adjacent feature fusion module, called AFSF, expands the number of channels to fuse the feature channels of several adjacent convolutional layers in the first half of the AFSFusion, enhancing its ability to extract, transmit, and modulate feature information. We replace the original rectified linear unit (ReLU) with leaky ReLU to alleviate the problem of gradient disappearance and add a channel shuffling operation at the end of AFSF to facilitate information interaction capability between features. Concerning loss functions, we propose an adaptive weight adjustment (AWA) strategy to assign weight values to the corresponding pixels of the infrared (IR) and visible images in the fused images, according to the VGG16 gradient feature response of the IR and visible images. This strategy efficiently handles different scene contents. After normalization, the weight values are used as weighting coefficients for the two sets of images. The weighting coefficients are applied to three loss items simultaneously: mean square error (MSE), structural similarity (SSIM), and total variation (TV), resulting in clearer objects and richer texture detail in the fused images. We conducted a series of experiments on several benchmark databases, and the results demonstrate the effectiveness of the proposed network architecture and the superiority of the proposed network compared to other state-of-the-art fusion methods. It ranks first in several objective metrics, showing the best performance and exhibiting sharper and richer edges of specific targets, which is more in line with human visual perception. The remarkable enhancement in performance is ascribed to the proposed AFSF module and AWA strategy, enabling balanced feature extraction, fusion, and modulation of image features throughout the process.

A Feature Fusion Method with Guided Training for Classification Tasks

Feature fusion network based on few-shot fine-grained classification

GCFnet: Global Collaborative Fusion Network for Multispectral and Panchromatic Image Classification

GTMFuse: Group-Attention Transformer-Driven Multiscale Dense Feature-Enhanced Network for Infrared and Visible Image Fusion

On Fusing the Latent Deep CNN Feature for Image Classification.

FDGNet: A pair feature difference guided network for multimodal medical image fusion

FusionNet: A Convolution–Transformer Fusion Network for Hyperspectral Image Classification

AMFF-net: Adaptive Multi-Modal Feature Fusion Network for Image Classification

GLFuse: A Global and Local Four-Branch Feature Extraction Network for Infrared and Visible Image Fusion

A Novel Feature Fusion with Self-adaptive Weight Method Based on Deep Learning for Image Classification

Infrared and Visible Image Fusion Based on Multiclassification Adversarial Mechanism in Feature Space

Feature-Decision Level Collaborative Fusion Network for Hyperspectral and LiDAR Classification

FDNet: An end-to-end fusion decomposition network for infrared and visible images

PointFusionNet: Point Feature Fusion Network for 3D Point Clouds Analysis

AFSFusion: an Adjacent Feature Shuffle Combination Network for Infrared and Visible Image Fusion

Feature Fusion Network for Long-Tailed Visual Recognition

DGFusion: an Effective Dynamic Generalizable Network for Infrared and Visible Image Fusion

A Global-Local Feature Adaptive Fusion Network for Image Scene Classification

A Feature Fusion Method for Feature Extraction

MFTCFNet: Infrared and Visible Image Fusion Network Based on Multi-Layer Feature Tightly Coupled

A Late Fusion Approach for Harnessing Multi-Cnn Model High-Level Features