Abstract:Hyperspectral images (HSIs) contain rich spatial and spectral detail information, while light detection and ranging (LiDAR) data can provide the elevation information. Thus, the fusion of HSI and LiDAR data can help in more accurate image classification, which becomes a hot research topic. However, it is difficult to capture complex local and global spatial-spectral associations; meanwhile, how to build an effective interaction between multimodal data is another important issue. To this end, a novel global-local transformer network (GLT-Net) is proposed for the joint classification of HSI and LiDAR data, in this article. The main idea is to fully exploit the advantage of the convolution operator in characterizing locally correlated features and the promising capability of transformer architecture in learning longrange dependencies. Moreover, multiscale feature fusion and probabilistic decision fusion strategies are also designed in one framework, to further improve the classification performance. Here, the proposed GLT-Net mainly consists of multiscale local spatial feature learning, global spectral feature learning, and global-local feature fusion classification. In specific, multimodal image cubes of different sizes are first extracted and sent into convolutional neural networks (CNNs) to learn local spatial features, which is followed by multimodal information propagation and spatial-attention-guided multiscale feature fusion. Afterward, by considering spectral feature channels from a sequential perspective, vision transformers are introduced to model the global spectral dependencies. Finally, multiple class estimations based on local and global features are integrated via a probabilistic decision fusion strategy. In this way, complementary information of multimodal data and local/global spectral-spatial information can be fully mined and jointly used. Extensive experiments on three popular HSI and LiDAR datasets demonstrate that the proposed method performs superiority over the state-of-the-art methods. The source code of the proposed method will be made publicly available at https://github.com/Ding-Kexin/GLT-Net.

Hierarchical local global transformer for point clouds analysis

LFT-Net: Local Feature Transformer Network for Point Clouds Analysis

EGCT: Enhanced Graph Convolutional Transformer for 3D Point Cloud Representation Learning

GTNet: Graph Transformer Network for 3D Point Cloud Classification and Semantic Segmentation

Local-to-Global Self-Attention in Vision Transformers

3DCTN: 3D Convolution-Transformer Network for Point Cloud Classification

Local Transformer Network on 3D Point Cloud Semantic Segmentation

Multi-Scale Geometric Feature Extraction and Global Transformer for Real-World Indoor Point Cloud Analysis

D2T-Net: A dual-domain transformer network exploiting spatial and channel dimensions for semantic segmentation of urban mobile laser scanning point clouds

Global–Local Transformer Network for HSI and LiDAR Data Joint Classification

Graph Transformer for 3d Point Clouds Classification and Semantic Segmentation

3DPCT: 3D Point Cloud Transformer with Dual Self-attention

LGLFormer: Local-global Lifting Transformer for Remote Sensing Scene Parsing

A Hierarchical Spatial Transformer for Massive Point Samples in Continuous Space

LSLPCT: an Enhanced Local Semantic Learning Transformer for 3-D Point Cloud Analysis

Local–Global Transformer Neural Network for Temporal Action Segmentation

Stratified transformer for 3d point cloud segmentation

HiTPR: Hierarchical Transformer for Place Recognition in Point Cloud

Spatial Transformer for 3D Point Clouds

APPT : Asymmetric Parallel Point Transformer for 3D Point Cloud Understanding

PAT: Point Cloud Analysis with Local Filter Embedding in Transformer