Abstract:The joint of multi-source remote sensing (RS) data for land cover classification has become a popular research topic. Although studies have shown that the fusion of multi-source data can improve the accuracy of classification, the current limitation lies in the inadequate exploitation of information, resulting in spectral confusion categories overlapping, and varying visual differences within the same category. To address these problems, this paper proposes a multi-scale spectral-spatial attention residual fusion network (MSSARFNet) that aims to enhance the classification performance of multi-source RS data through effective spectral-spatial feature extraction and fusion. Specifically, three modules are designed: the multi-scale spectral attention residual module (MSpeARM), the multi-scale spatial attention residual module (MSpaARM) and multi-scale convolution fusion (MCF) module. Firstly, we divide the channels of the features into multiple paths and apply spectral-spatial attention mechanisms on each path. To further enhance connectivity, convolutional operation is utilized to establish connections between different paths, thus forming MSpeARM and MSpaARM. The MSpeARM suppresses redundant features and enhances effective features along the channel dimension to better differentiate spectral-confused categories. The MSpaARM highlights that different visual patterns of objects in the same category can mutually reinforce each other by weighting all positional features, regardless of their spatial differences. Secondly, to fuse these two sets of features, the MCF module is designed to learn multi-level semantic features and enhance fusion at a granular level. Experimental evaluations on three RS datasets demonstrate that the proposed method achieves excellent classification performance, indicating the effectiveness of MSSARFNet. The source code of the proposed method will be available at https://github.com/wangxu0210/MSSARFNet .

Triplet Attention Feature Fusion Network for SAR and Optical Image Land Cover Classification.

Hyperspectral and Multispectral Image Fusion Based on Deep Attention Network.

Feature Fusion Classification for Optical Image and SAR Image Based on Spatial-spectral Attention

MFFnet: Multimodal Feature Fusion Network for Synthetic Aperture Radar and Optical Image Land Cover Classification

Triple disentangled network with dual attention for remote sensing image fusion

DAFCNN: A Dual-Channel Feature Extraction and Attention Feature Fusion Convolution Neural Network for SAR Image and MS Image Fusion

Multimodal Bilinear Fusion Network with Second-Order Attention-Based Channel Selection for Land Cover Classification

Cross-Modality Attentive Feature Fusion for Object Detection in Multispectral Remote Sensing Imagery

Learning SAR-Optical Cross Modal Features for Land Cover Classification

A Triplet Semisupervised Deep Network for Fusion Classification of Hyperspectral and LiDAR Data

Multi-Scale Spectral-Spatial Attention Residual Fusion Network for Multi-Source Remote Sensing Data Classification

Hierarchical Attention and Parallel Filter Fusion Network for Multisource Data Classification

Attention Fusion of Transformer-Based and Scale-Based Method for Hyperspectral and LiDAR Joint Classification

Collaborative Attention-Based Heterogeneous Gated Fusion Network for Land Cover Classification

Hierarchical Attention and Parallel Filter Fusion Network for Multi-Source Data Classification

CFNet: A Cross Fusion Network for Joint Land Cover Classification Using Optical and SAR Images

Hyperspectral and Multi-source Heterogeneous Data Fusion Classification Based on Multiscale Multi-source Interaction Attention Network

Multiscale Channel Attention Network for Infrared and Visible Image Fusion.

Feature Fusion Network Model Based on Dual Attention Mechanism for Hyperspectral Image Classification

Multi-scale unsupervised network for infrared and visible image fusion based on joint attention mechanism

Multibranch Feature Fusion Network with Self- and Cross-Guided Attention for Hyperspectral and LiDAR Classification