Abstract:Transformer has been widely applied in image processing tasks as a substitute for convolutional neural networks (CNNs) for feature extraction due to its superiority in global context modeling and flexibility in model generalization. However, the existing transformer-based methods for semantic segmentation of remote sensing (RS) images are still with several limitations, which can be summarized into two main aspects: 1) the transformer encoder is generally combined with CNN-based decoder, leading to inconsistency in feature representations; and 2) the strategies for global and local context information utilization are not sufficiently effective. Therefore, in this article, a global-local transformer segmentor (GLOTS) framework is proposed for the semantic segmentation of RS images to acquire consistent feature representations by adopting transformers for both encoding and decoding, in which a masked image modeling (MIM) pretrained transformer encoder is adopted to learn semantic-rich representations of input images and a multiscale global-local transformer decoder is designed to fully exploit the global and local features. Specifically, the transformer decoder uses a feature separation-aggregation module (FSAM) to utilize the feature adequately at different scales and adopts a global-local attention module (GLAM) containing global attention block (GAB) and local attention block (LAB) to capture the global and local context information, respectively. Furthermore, a learnable progressive upsampling strategy (LPUS) is proposed to restore the resolution progressively, which can flexibly recover the fine-grained details in the upsampling process. The experiment results on the three benchmark RS datasets demonstrate that the proposed GLOTS is capable of achieving better performance with some state-of-the-art methods, and the superiority of the proposed framework is also verified by ablation studies. The code will be available at https://github.com/lyhnsn/GLOTS.

Radial Transformer for Large-Scale Outdoor LiDAR Point Cloud Semantic Segmentation

A Transformer-based Real-time LiDAR Semantic Segmentation Method for Restricted Mobile Devices

Spherical Transformer for LiDAR-based 3D Recognition

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation

Improving Radial Imbalances with Hybrid Voxelization and RadialMix for LiDAR 3D Semantic Segmentation

Rethinking Transformers for Semantic Segmentation of Remote Sensing Images.

LEST: Large-scale LiDAR Semantic Segmentation with Transformer

Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception

3D Semantic Segmentation Using Deep Learning for Large-Scale Indoor Point Cloud

Local Transformer Network on 3D Point Cloud Semantic Segmentation

Outdoor large-scene 3D point cloud reconstruction based on transformer

Multi-Scale Point-Wise Convolutional Neural Networks for 3D Object Segmentation From LiDAR Point Clouds in Large-Scale Environments

Regional-to-Local Point-Voxel Transformer for Large-Scale Indoor 3D Point Cloud Semantic Segmentation

D2T-Net: A dual-domain transformer network exploiting spatial and channel dimensions for semantic segmentation of urban mobile laser scanning point clouds

Transformer Meets Convolution: A Bilateral Awareness Network for Semantic Segmentation of Very Fine Resolution Urban Scene Images

Stratified Transformer for 3D Point Cloud Segmentation

Rethinking 3D LiDAR Point Cloud Segmentation

Real-Time Semantic Segmentation of LiDAR Point Clouds on Edge Devices for Unmanned Systems

Uplifting Range-View-based 3D Semantic Segmentation in Real-Time with Multi-Sensor Fusion

Hybrid CNN-LSTM Architecture for LiDAR Point Clouds Semantic Segmentation