Abstract:Learning with limited labelled data is a challenging problem in various applications, including remote sensing. Few-shot semantic segmentation is one approach that can encourage deep learning models to learn from few labelled examples for novel classes not seen during the training. The generalized few-shot segmentation setting has an additional challenge which encourages models not only to adapt to the novel classes but also to maintain strong performance on the training base classes. While previous datasets and benchmarks discussed the few-shot segmentation setting in remote sensing, we are the first to propose a generalized few-shot segmentation benchmark for remote sensing. The generalized setting is more realistic and challenging, which necessitates exploring it within the remote sensing context. We release the dataset augmenting OpenEarthMap with additional classes labelled for the generalized few-shot evaluation setting. The dataset is released during the OpenEarthMap land cover mapping generalized few-shot challenge in the L3D-IVU workshop in conjunction with CVPR 2024. In this work, we summarize the dataset and challenge details in addition to providing the benchmark results on the two phases of the challenge for the validation and test sets.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to perform Generalized Few - Shot Semantic Segmentation (GFSS) in remote sensing images. Specifically, the paper focuses on how to make the deep - learning model adapt to novel classes with only a small amount of labeled data, while maintaining high performance on base classes that have been seen during training. ### Main problems 1. **Learning with limited labeled data**: Labeled data in remote sensing images is usually very expensive and difficult to obtain. Therefore, how to use a small amount of labeled data for effective model training is an important challenge. 2. **Generalized few - shot segmentation**: Compared with the standard few - shot segmentation tasks, generalized few - shot segmentation requires the model not only to be able to handle novel classes, but also to ensure that its performance on base classes does not decline. This setting is more in line with the needs of practical application scenarios. 3. **Lack of dedicated benchmark datasets**: Although previous studies have explored few - shot segmentation in remote sensing images, there are no benchmark datasets and evaluation frameworks specifically for generalized few - shot segmentation. ### Solutions To solve the above problems, the authors made the following contributions: - **OEM - GFSS dataset**: Extended the existing OpenEarthMap dataset, added new fine - grained feature classes, and constructed a high - resolution land - cover mapping dataset containing 15 classes. This dataset is used to support the generalized few - shot segmentation task. - **Proposed the generalized few - shot segmentation benchmark for the first time**: This is the first benchmark dataset for generalized few - shot segmentation of remote sensing images, filling the gap in this field. - **Challenge and baseline model**: By holding the L3D - IVU workshop challenge, baseline models and evaluation results were provided, encouraging more researchers to participate and promoting the development of this field. ### Formula explanation When describing the model performance, the paper uses the Intersection over Union (IoU) as an evaluation metric and introduces the Weighted - Sum mIoU to comprehensively evaluate the performance of base classes and novel classes: \[ \text{Weighted - Sum mIoU} = 0.4\times m_{\text{base}}+ 0.6\times m_{\text{novel}} \] where \(m_{\text{base}}\) and \(m_{\text{novel}}\) represent the mean Intersection over Union of the base classes and the novel classes respectively. Through these measures, the paper aims to promote the development of few - shot learning techniques in remote sensing images, especially to improve the generalization ability of the model in the case of dealing with limited labeled data.

Generalized Few-Shot Semantic Segmentation in Remote Sensing: Challenge and Benchmark

Generalized Few-Shot Meets Remote Sensing: Discovering Novel Classes in Land Cover Mapping via Hybrid Semantic Segmentation Framework

Learnable Prompt for Few-Shot Semantic Segmentation in Remote Sensing Domain

Self-guided Few-shot Semantic Segmentation for Remote Sensing Imagery Based on Large Vision Models

Few-shot remote sensing image segmentation based on label propagation and open-set domain adaptation

GAMUS: A Geometry-aware Multi-modal Semantic Segmentation Benchmark for Remote Sensing Data

Holistic Mutual Representation Enhancement for Few-Shot Remote Sensing Segmentation

Semi-Supervised Contrastive Learning for Few-Shot Segmentation of Remote Sensing Images

Iterative Few-shot Semantic Segmentation from Image Label Text

Embedding Generalized Semantic Knowledge into Few-Shot Remote Sensing Segmentation

Not Just Learning from Others but Relying on Yourself: A New Perspective on Few-Shot Segmentation in Remote Sensing

SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images

Category-sensitive Semi-Supervised Semantic Segmentation Framework for Land-Use/land-cover Mapping with Optical Remote Sensing Images

Self-Correlation and Cross-Correlation Learning for Few-Shot Remote Sensing Image Semantic Segmentation

Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges

Few-shot 3D LiDAR Semantic Segmentation for Autonomous Driving

Research Progress on Few-Shot Learning for Remote Sensing Image Interpretation

Semi-Supervised Semantic Segmentation in Earth Observation: The MiniFrance Suite, Dataset Analysis and Multi-task Network Study

Text2Seg: Remote Sensing Image Semantic Segmentation via Text-Guided Visual Foundation Models

WildScenes: A benchmark for 2D and 3D semantic segmentation in large-scale natural environments

Active Learning for Improved Semi-Supervised Semantic Segmentation in Satellite Images