Abstract:Infrared and visible image fusion aims to extract complementary features to synthesize a single fused image. Many methods employ convolutional neural networks (CNNs) to extract local features due to its translation invariance and locality. However, CNNs fail to consider the image's non-local self-similarity (NLss), though it can expand the receptive field by pooling operations, it still inevitably leads to information loss. In addition, the transformer structure extracts long-range dependence by considering the correlativity among all image patches, leading to information redundancy of such transformer-based methods. However, graph representation is more flexible than grid (CNN) or sequence (transformer structure) representation to address irregular objects, and graph can also construct the relationships among the spatially repeatable details or texture with far-space distance. Therefore, to address the above issues, it is significant to convert images into the graph space and thus adopt graph convolutional networks (GCNs) to extract NLss. This is because the graph can provide a fine structure to aggregate features and propagate information across the nearest vertices without introducing redundant information. Concretely, we implement a cascaded NLss extraction pattern to extract NLss of intra- and inter-modal by exploring interactions of different image pixels in intra- and inter-image positional distance. We commence by preforming GCNs on each intra-modal to aggregate features and propagate information to extract independent intra-modal NLss. Then, GCNs are performed on the concatenate intra-modal NLss features of infrared and visible images, which can explore the cross-domain NLss of inter-modal to reconstruct the fused image. Ablation studies and extensive experiments illustrates the effectiveness and superiority of the proposed method on three datasets.

On the Exploration of Convolutional Fusion Networks for Visual Recognition.

Fusion that matters: convolutional fusion networks for visual recognition

Fusion of Low-Illuminance Visible and Near-Infrared Images Based on Convolutional Neural Networks

Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution

Multi-Scale Feature Fusion of Covariance Pooling Networks for Fine-Grained Visual Recognition.

Aerial Scene Classification Via Multilevel Fusion Based on Deep Convolutional Neural Networks.

Delving into Fully Convolutional Networks Activations for Visual Recognition.

AMFF-net: Adaptive Multi-Modal Feature Fusion Network for Image Classification

IFCNN: A General Image Fusion Framework Based on Convolutional Neural Network.

A Late Fusion Approach for Harnessing Multi-Cnn Model High-Level Features

Self-Adaption Multi-classifier Fusion Networks for Image Recognition

Graph Representation Learning for Infrared and Visible Image Fusion

Fast Graph Convolution Network Based Multi-label Image Recognition via Cross-modal Fusion

Hybrid-Order and Multi-Stream Convolutional Neural Network for Fine-Grained Visual Recognition

CMFuse: Cross-Modal Features Mixing Via Convolution and MLP for Infrared and Visible Image Fusion

Multilevel Features Fusion In Deep Convolutional Neural Networks

CCAFusion: Cross-Modal Coordinate Attention Network for Infrared and Visible Image Fusion

ResDNet: Efficient Dense Multi-Scale Representations with Residual Learning for High-Level Vision Tasks

Multilayer Feature Descriptors Fusion CNN Models for Fine-Grained Visual Recognition

Improving the Performance of Convolutional Neural Networks by Fusing Low-Level Features with Different Scales in the Preceding Stage

Infrared and Visible Image Fusion with Convolutional Neural Networks.