Abstract:The effective fusion of multisource data helps to improve the performance of land cover classification. Most existing convolutional neural network (CNN)-based methods adopt an early/late fusion strategy to fuse the low-/high-level features for classification, which still has two inherent challenges: 1) the conventional convolution operation performs a weighted average operation on each pixel in the receptive field, which will reduce the discriminability of the center pixel due to the influence of the interference pixels and 2) the spatial-spectral features of the hyperspectral image (HSI), the elevation features of light detection and ranging (LiDAR), and the complementary features between the multimodal data are not fully exploited, which results in the reduction of classification accuracy. In this article, an effective multibranch feature fusion network with self- and cross-guided attention (MB2FscgaNet) is proposed for the joint classification of LiDAR and HSI. The main concern of this article is how to accurately estimate more effective spectral-spatial-elevation features and yield more effective transfer in the network. Specifically, MB2FscgaNet adopts a multibranch feature fusion architecture to fully exploit the hierarchical features from LiDAR and HSI level by level. At each level of the network, a self- and cross-guided attention (SCGA) is developed to assign a higher weight to interesting areas and channels of LiDAR and HSI feature maps to obtain refined spectral-spatial-elevation features and provide complementary information cross-guidance between LiDAR and HS. We further designed a spectral supplement module (SeSuM) to improve the discriminative ability of the center pixel. Comparative classification results and ablation studies demonstrate that the proposed MB2FscgaNet achieves competitive performance against state-of-the-art methods.

Feature Correlation Analysis of Two-Branch Convolutional Networks for Multi-Source Image Classification.

Research on Image Classification Method of Features of Combinatorial Convolution

Dynamic Convolution Covariance Network Using Multi-Scale Feature Fusion for Remote Sensing Scene Image Classification

Multi-Spectral RGB-NIR Image Classification Using Double-Channel CNN

Multi-source remote sensing image classification based on two-channel densely connected convolutional networks

Multisource Remote Sensing Data Classification Based on Convolutional Neural Network

A Multi-Branch Feature Fusion Strategy Based on an Attention Mechanism for Remote Sensing Image Scene Classification

Dual-Branch Feature Fusion Network Based Cross-Modal Enhanced CNN and Transformer for Hyperspectral and LiDAR Classification

A Multi-branch Feature Fusion Model Based on Convolutional Neural Network for Hyperspectral Remote Sensing Image Classification

A Two-Branch Network with Semi-Supervised Learning for Hyperspectral Classification

Branch Feature Fusion Convolution Network for Remote Sensing Scene Classification

Image Co-segmentation with Multi-Scale Dual-Cross Correlation Network

Synthetical application of multi-feature map detection and multi-branch convolution

Fusion Of Hyperspectral And Lidar Data Based On Dual-Branch Convolutional Neural Network

Feature Correlation Loss in Convolutional Neural Networks for Image Classification

DCA-GCN: a Dual-Branching Channel Attention and Graph Convolution Network for Multi-Label Remote Sensing Image Classification

A Collaborative Correlation-Matching Network for Multimodality Remote Sensing Image Classification.

Multi-branch Selection Fusion Fine-Grained Classification Algorithm Based on Coordinate Attention Localization

Multi-branch Prediction Network for Multi-label Social Image Classification.

Multibranch Feature Fusion Network with Self- and Cross-Guided Attention for Hyperspectral and LiDAR Classification

CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion