Abstract:Semantic segmentation of remote sensing (RS) images is vital in various practical applications, including urban construction planning, natural disaster monitoring, and land resources investigation. However, RS images are captured by airplanes or satellites at high altitudes and long distances, resulting in ground objects of the same category being scattered in various corners of the image. Moreover, objects of different sizes appear simultaneously in RS images. For example, some objects occupy a large area in urban scenes, while others only have small regions. Technically, the above two universal situations pose significant challenges to the segmentation with a high quality for RS images. Based on these observations, this paper proposes a Mask2Former with an improved query (IQ2Former) for this task. The fundamental motivation behind the IQ2Former is to enhance the capability of the query of Mask2Former by exploiting the characteristics of RS images well. First, we propose the Query Scenario Module (QSM), which aims to learn and group the queries from feature maps, allowing the selection of distinct scenarios such as the urban and rural areas, building clusters, and parking lots. Second, we design the query position module (QPM), which is developed to assign the image position information to each query without increasing the number of parameters, thereby enhancing the model's sensitivity to small targets in complex scenarios. Finally, we propose the query attention module (QAM), which is constructed to leverage the characteristics of query attention to extract valuable features from the preceding queries. Being positioned between the duplicated transformer decoder layers, QAM ensures the comprehensive utilization of the supervisory information and the exploitation of those fine-grained details. Architecturally, the QSM, QPM, and QAM as well as an end-to-end model are assembled to achieve high-quality semantic segmentation. In comparison to the classical or state-of-the-art models (FCN, PSPNet, DeepLabV3+, OCRNet, UPerNet, MaskFormer, Mask2Former), IQ2Former has demonstrated exceptional performance across three publicly challenging remote-sensing image datasets, 83.59 mIoU on the Vaihingen dataset, 87.89 mIoU on Potsdam dataset, and 56.31 mIoU on LoveDA dataset. Additionally, overall accuracy, ablation experiment, and visualization segmentation results all indicate IQ2Former validity.

DropQueries: A Simple Way to Discover Comprehensive Segment Representations

DQFormer: Towards Unified LiDAR Panoptic Segmentation with Decoupled Queries

Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation

Dynamic Focus-aware Positional Queries for Semantic Segmentation

Learning Equivariant Segmentation with Instance-Unique Querying

Multi-query and Multi-Level Enhanced Network for Semantic Segmentation

Query Learning of Both Thing and Stuff for Panoptic Segmentation.

QueryInst: Parallelly Supervised Mask Query for Instance Segmentation

SOLQ: Segmenting Objects by Learning Queries

Exploring Dual Representations in Large-Scale Point Clouds: A Simple Weakly Supervised Semantic Segmentation Framework

Mask2Former with Improved Query for Semantic Segmentation in Remote-Sensing Images

Learning Task-Aligned Mask Query for Instance Segmentation

Mask Matching Transformer for Few-Shot Segmentation

LESS: Label-Efficient and Single-Stage Referring 3D Segmentation

Position-Guided Point Cloud Panoptic Segmentation Transformer

QTSeg: A Query Token-Based Architecture for Efficient 2D Medical Image Segmentation

Adaptive Query Selection for Camouflaged Instance Segmentation

Devil is in the Queries: Advancing Mask Transformers for Real-world Medical Image Segmentation and Out-of-Distribution Localization

3D-QueryIS: A Query-based Framework for 3D Instance Segmentation

Few-shot Segmentation with Duplex Network and Attention Augmented Module.