Abstract:Hyperspectral images (HSIs) contain rich spatial and spectral detail information, while light detection and ranging (LiDAR) data can provide the elevation information. Thus, the fusion of HSI and LiDAR data can help in more accurate image classification, which becomes a hot research topic. However, it is difficult to capture complex local and global spatial-spectral associations; meanwhile, how to build an effective interaction between multimodal data is another important issue. To this end, a novel global-local transformer network (GLT-Net) is proposed for the joint classification of HSI and LiDAR data, in this article. The main idea is to fully exploit the advantage of the convolution operator in characterizing locally correlated features and the promising capability of transformer architecture in learning longrange dependencies. Moreover, multiscale feature fusion and probabilistic decision fusion strategies are also designed in one framework, to further improve the classification performance. Here, the proposed GLT-Net mainly consists of multiscale local spatial feature learning, global spectral feature learning, and global-local feature fusion classification. In specific, multimodal image cubes of different sizes are first extracted and sent into convolutional neural networks (CNNs) to learn local spatial features, which is followed by multimodal information propagation and spatial-attention-guided multiscale feature fusion. Afterward, by considering spectral feature channels from a sequential perspective, vision transformers are introduced to model the global spectral dependencies. Finally, multiple class estimations based on local and global features are integrated via a probabilistic decision fusion strategy. In this way, complementary information of multimodal data and local/global spectral-spatial information can be fully mined and jointly used. Extensive experiments on three popular HSI and LiDAR datasets demonstrate that the proposed method performs superiority over the state-of-the-art methods. The source code of the proposed method will be made publicly available at https://github.com/Ding-Kexin/GLT-Net.

A Joint Convolutional Cross ViT Network for Hyperspectral and Light Detection and Ranging Fusion Classification

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification

Cross Attention-Based Multi-Scale Convolutional Fusion Network for Hyperspectral and LiDAR Joint Classification

Joint Classification of Hyperspectral and LiDAR Data Using a Hierarchical CNN and Transformer

Convolution Transformer Fusion Splicing Network for Hyperspectral Image Classification

Collaborative Classification of Hyperspectral and Visible Images with Convolutional Neural Network

Multisource Remote Sensing Data Classification Based on Convolutional Neural Network

Information Fusion for Classification of Hyperspectral and LiDAR Data Using IP-CNN

Joint Classification of Hyperspectral Images and LiDAR Data Based on Dual-Branch Transformer

Multiple Information Collaborative Fusion Network for Joint Classification of Hyperspectral and LiDAR Data

Dual-Branch Feature Fusion Network Based Cross-Modal Enhanced CNN and Transformer for Hyperspectral and LiDAR Classification

Combined Classification of Hyperspectral and LiDAR Data Based on Dual-Channel Cross-Transformer.

Bridging CNN and Transformer With Cross-Attention Fusion Network for Hyperspectral Image Classification

Hybrid FusionNet: A Hybrid Feature Fusion Framework for Multi-Source High-Resolution Remote Sensing Image Classification

Hybrid FusionNet: A Hybrid Feature Fusion Framework for Multisource High-Resolution Remote Sensing Image Classification

Classification of hyperspectral and LiDAR data by transformer-based enhancement

Multi-layer feature fusion for hyperspectral image classification

Convolution-Transformer Adaptive Fusion Network for Hyperspectral Image Classification

Multiple Vision Architectures-Based Hybrid Network for Hyperspectral Image Classification

Global–Local Transformer Network for HSI and LiDAR Data Joint Classification