Abstract:Transformer has been widely applied in image processing tasks as a substitute for convolutional neural networks (CNNs) for feature extraction due to its superiority in global context modeling and flexibility in model generalization. However, the existing transformer-based methods for semantic segmentation of remote sensing (RS) images are still with several limitations, which can be summarized into two main aspects: 1) the transformer encoder is generally combined with CNN-based decoder, leading to inconsistency in feature representations; and 2) the strategies for global and local context information utilization are not sufficiently effective. Therefore, in this article, a global-local transformer segmentor (GLOTS) framework is proposed for the semantic segmentation of RS images to acquire consistent feature representations by adopting transformers for both encoding and decoding, in which a masked image modeling (MIM) pretrained transformer encoder is adopted to learn semantic-rich representations of input images and a multiscale global-local transformer decoder is designed to fully exploit the global and local features. Specifically, the transformer decoder uses a feature separation-aggregation module (FSAM) to utilize the feature adequately at different scales and adopts a global-local attention module (GLAM) containing global attention block (GAB) and local attention block (LAB) to capture the global and local context information, respectively. Furthermore, a learnable progressive upsampling strategy (LPUS) is proposed to restore the resolution progressively, which can flexibly recover the fine-grained details in the upsampling process. The experiment results on the three benchmark RS datasets demonstrate that the proposed GLOTS is capable of achieving better performance with some state-of-the-art methods, and the superiority of the proposed framework is also verified by ablation studies. The code will be available at https://github.com/lyhnsn/GLOTS.

A Unified Efficient Pyramid Transformer for Semantic Segmentation

SDPT: Semantic-Aware Dimension-Pooling Transformer for Image Segmentation

Pyramid Fusion Transformer for Semantic Segmentation

PSSD-Transformer: Powerful Sparse Spike-Driven Transformer for Image Semantic Segmentation

Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers

Rethinking Transformers for Semantic Segmentation of Remote Sensing Images.

Efficient Hybrid Transformer: Learning Global-local Context for Urban Sence Segmentation

Dual-resolution Transformer Combined with Multi-Layer Separable Convolution Fusion Network for Real-Time Semantic Segmentation

Graph-Segmenter: Graph Transformer with Boundary-aware Attention for Semantic Segmentation

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers

Efficient pyramid context encoding and feature embedding for semantic segmentation

Fully Transformer Networks for Semantic Image Segmentation

UNetFormer: A UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery

Efficient Transformer for Remote Sensing Image Segmentation

Enhanced Feature Pyramid Network for Semantic Segmentation.

Semantic Segmentation Based on Vision Transformer Via Interactive Attention

ESMS-Net: Enhancing Semantic-Mask Segmentation Network with Pyramid Atrousformer for Remote Sensing Image

P2AT: Pyramid Pooling Axial Transformer for Real-time Semantic Segmentation

Refined Division Features Based on Transformer for Semantic Image Segmentation

Multi-Scale Transformer with Explicit Boundary Constraint for Semantic Segmentation

PyramidMamba: Rethinking Pyramid Feature Fusion with Selective Space State Model for Semantic Segmentation of Remote Sensing Imagery