Abstract:Semantic segmentation of remote sensing (RS) images is vital in various practical applications, including urban construction planning, natural disaster monitoring, and land resources investigation. However, RS images are captured by airplanes or satellites at high altitudes and long distances, resulting in ground objects of the same category being scattered in various corners of the image. Moreover, objects of different sizes appear simultaneously in RS images. For example, some objects occupy a large area in urban scenes, while others only have small regions. Technically, the above two universal situations pose significant challenges to the segmentation with a high quality for RS images. Based on these observations, this paper proposes a Mask2Former with an improved query (IQ2Former) for this task. The fundamental motivation behind the IQ2Former is to enhance the capability of the query of Mask2Former by exploiting the characteristics of RS images well. First, we propose the Query Scenario Module (QSM), which aims to learn and group the queries from feature maps, allowing the selection of distinct scenarios such as the urban and rural areas, building clusters, and parking lots. Second, we design the query position module (QPM), which is developed to assign the image position information to each query without increasing the number of parameters, thereby enhancing the model's sensitivity to small targets in complex scenarios. Finally, we propose the query attention module (QAM), which is constructed to leverage the characteristics of query attention to extract valuable features from the preceding queries. Being positioned between the duplicated transformer decoder layers, QAM ensures the comprehensive utilization of the supervisory information and the exploitation of those fine-grained details. Architecturally, the QSM, QPM, and QAM as well as an end-to-end model are assembled to achieve high-quality semantic segmentation. In comparison to the classical or state-of-the-art models (FCN, PSPNet, DeepLabV3+, OCRNet, UPerNet, MaskFormer, Mask2Former), IQ2Former has demonstrated exceptional performance across three publicly challenging remote-sensing image datasets, 83.59 mIoU on the Vaihingen dataset, 87.89 mIoU on Potsdam dataset, and 56.31 mIoU on LoveDA dataset. Additionally, overall accuracy, ablation experiment, and visualization segmentation results all indicate IQ2Former validity.

Maskformer with Improved Encoder-Decoder Module for Semantic Segmentation of Fine-Resolution Remote Sensing Images.

SeMask-Mask2Former: A Semantic Segmentation Model for High Resolution Remote Sensing Images

Rethinking Transformers for Semantic Segmentation of Remote Sensing Images.

MMT: Mixed-Mask Transformer for Remote Sensing Image Semantic Segmentation

ESMS-Net: Enhancing Semantic-Mask Segmentation Network with Pyramid Atrousformer for Remote Sensing Image

Mask2Former with Improved Query for Semantic Segmentation in Remote-Sensing Images

MP-Former: Mask-Piloted Transformer for Image Segmentation

Enhancing Mask Transformer with Auxiliary Convolution Layers for Semantic Segmentation

Mask-R-FCN: A Deep Fusion Network for Semantic Segmentation.

BiTSRS: A Bi-Decoder Transformer Segmentor for High-Spatial-Resolution Remote Sensing Images

Masked-attention Mask Transformer for Universal Image Segmentation

ACTNet: A Dual-Attention Adapter with a CNN-Transformer Network for the Semantic Segmentation of Remote Sensing Imagery

Pyramid Fusion Transformer for Semantic Segmentation

Mask Encoding: A General Instance Mask Representation for Object Segmentation

Learning Content-enhanced Mask Transformer for Domain Generalized Urban-Scene Segmentation

Transformer and CNN Hybrid Deep Neural Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery

MUSTER: A Multi-scale Transformer-based Decoder for Semantic Segmentation

SeMask: Semantically Masked Transformers for Semantic Segmentation

Mask Transfiner for High-Quality Instance Segmentation

When zero-padding position encoding encounters linear space reduction attention: an efficient semantic segmentation Transformer of remote sensing images

Remote Sensing Image Semantic Segmentation Based on Fusion of Transformer and Lightweight Deeplabv3+