Abstract:The effective fusion of multisource data helps to improve the performance of land cover classification. Most existing convolutional neural network (CNN)-based methods adopt an early/late fusion strategy to fuse the low-/high-level features for classification, which still has two inherent challenges: 1) the conventional convolution operation performs a weighted average operation on each pixel in the receptive field, which will reduce the discriminability of the center pixel due to the influence of the interference pixels and 2) the spatial-spectral features of the hyperspectral image (HSI), the elevation features of light detection and ranging (LiDAR), and the complementary features between the multimodal data are not fully exploited, which results in the reduction of classification accuracy. In this article, an effective multibranch feature fusion network with self- and cross-guided attention (MB2FscgaNet) is proposed for the joint classification of LiDAR and HSI. The main concern of this article is how to accurately estimate more effective spectral-spatial-elevation features and yield more effective transfer in the network. Specifically, MB2FscgaNet adopts a multibranch feature fusion architecture to fully exploit the hierarchical features from LiDAR and HSI level by level. At each level of the network, a self- and cross-guided attention (SCGA) is developed to assign a higher weight to interesting areas and channels of LiDAR and HSI feature maps to obtain refined spectral-spatial-elevation features and provide complementary information cross-guidance between LiDAR and HS. We further designed a spectral supplement module (SeSuM) to improve the discriminative ability of the center pixel. Comparative classification results and ablation studies demonstrate that the proposed MB2FscgaNet achieves competitive performance against state-of-the-art methods.

Multibranch Feature Difference Learning Network for Cross-Spectral Image Patch Matching

AFD-Net: Aggregated Feature Difference Learning for Cross-Spectral Image Patch Matching

Efficient Feature Relation Learning Network for Cross-Spectral Image Patch Matching

Feature Interaction Learning Network for Cross-Spectral Image Patch Matching

Element-Wise Feature Relation Learning Network for Cross-Spectral Image Patch Matching

Why and How: Knowledge-Guided Learning for Cross-Spectral Image Patch Matching

Cross-Spectral Image Patch Matching By Learning Features Of The Spatially Connected Patches In A Shared Space

Metric Learning for 2D Image Patch and 3D Point Cloud Volume Matching.

SSML-QNet: Scale-Separative Metric Learning Quadruplet Network for Multi-Modal Image Patch Matching

Multi-Relation Attention Network for Image Patch Matching.

A Novel Neural Network for Remote Sensing Image Matching

Deep Continuous Matching Network for More Robust Multi-Modal Remote Sensing Image Patch Matching

Deep Feature Correlation Learning for Multi-Modal Remote Sensing Image Registration

Relational Representation Learning Network for Cross-Spectral Image Patch Matching

Deep modality independent descriptor learning for optical and sar image patch matching

Metric networks for enhanced perception of non-local semantic information

A Collaborative Correlation-Matching Network for Multimodality Remote Sensing Image Classification.

Learning multi-domain feature relation for visible and Long-wave Infrared image patch matching

Multifeature Alignment and Matching Network for SAR and Optical Image Registration

Multibranch Feature Fusion Network with Self- and Cross-Guided Attention for Hyperspectral and LiDAR Classification

Deep Siamese Multi-level Feature Network for VIS and NIR Image Matching