Abstract:In recent years, deep learning-based multi-source data fusion, e.g., hyperspectral image (HSI) and light detection and ranging (LiDAR) data fusion, has gained significant attention in the field of remote sensing. However, the traditional convolutional neural network fusion techniques always provide poor extraction of discriminative spatial–spectral features from diversified land covers and overlook the correlation and complementarity between different data sources. Furthermore, the mere act of stacking multi-source feature embeddings fails to represent the deep semantic relationships among them. In this paper, we propose a cross attention-based multi-scale convolutional fusion network for HSI-LiDAR joint classification. It contains three major modules: spatial–elevation–spectral convolutional feature extraction module (SESM), cross attention fusion module (CAFM), and classification module. In the SESM, improved multi-scale convolutional blocks are utilized to extract features from HSI and LiDAR to ensure discriminability and comprehensiveness in diversified land cover conditions. Spatial and spectral pseudo-3D convolutions, pointwise convolutions, residual aggregation, one-shot aggregation, and parameter-sharing techniques are implemented in the module. In the CAFM, a self-designed local-global cross attention block is utilized to collect and integrate relationships of the feature embeddings and generate joint semantic representations. In the classification module, average polling, dropout, and linear layers are used to map the fused semantic representations to the final classification results. The experimental evaluations on three public HSI-LiDAR datasets demonstrate the competitiveness of the proposed network in comparison with state-of-the-art methods.

A New Multi-Level Attention Feature Fusion Method for Hyperspectral and Lidar Data Joint Classification

Hyperspectral and Multispectral Image Fusion Based on Deep Attention Network.

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

Joint Classification of Hyperspectral and LiDAR Data Based on Heterogeneous Attention Feature Fusion Network

Cross Attention-Based Multi-Scale Convolutional Fusion Network for Hyperspectral and LiDAR Joint Classification

Multi-Scale Feature Fusion for Hyperspectral and Lidar Data Joint Classification

Hyperspectral and Multi-source Heterogeneous Data Fusion Classification Based on Multiscale Multi-source Interaction Attention Network

Joint Classification of Hyperspectral and Lidar Data Using Cross-Modal Hierarchical Frequency Fusion Network

LiDAR-Guided Cross-Attention Fusion for Hyperspectral Band Selection and Image Classification

Discriminative feature extraction and fusion for classification of hyperspectral and lidar data

Attention Fusion of Transformer-Based and Scale-Based Method for Hyperspectral and LiDAR Joint Classification

Joint Classification of Hyperspectral and LiDAR Data Based on Position-Channel Cooperative Attention Network

Hierarchical Attention and Parallel Filter Fusion Network for Multi-Source Data Classification

Joint Classification of Hyperspectral Images and LiDAR Data Based on Dual-Branch Transformer

Joint Classification of Hyperspectral and LiDAR Data Using Height Information Guided Hierarchical Fusion-and-Separation Network

Classification of hyperspectral and LiDAR data by transformer-based enhancement

Hyperspectral and Lidar Data Classification Based on Linear Self-Attention.

Combined Classification of Hyperspectral and LiDAR Data Based on Dual-Channel Cross-Transformer.

A Multiscale Dual-Branch Feature Fusion and Attention Network for Hyperspectral Images Classification

Hierarchical Attention and Parallel Filter Fusion Network for Multisource Data Classification