Abstract:This paper studies a practical domain adaptive (DA) semantic segmentation problem where only pseudo-labeled target data is accessible through a black-box model. Due to the domain gap and label shift between two domains, pseudo-labeled target data contains mixed closed-set and open-set label noises. In this paper, we propose a simplex noise transition matrix (SimT) to model the mixed noise distributions in DA semantic segmentation, and leverage SimT to handle open-set label noise and enable novel target recognition. When handling open-set noises, we formulate the problem as estimation of SimT. By exploiting computational geometry analysis and properties of segmentation, we design four complementary regularizers, i.e. volume regularization, anchor guidance, convex guarantee, and semantic constraint, to approximate the true SimT. Specifically, volume regularization minimizes the volume of simplex formed by rows of the non-square SimT, ensuring outputs of model to fit into the ground truth label distribution. To compensate for the lack of open-set knowledge, anchor guidance, convex guarantee, and semantic constraint are devised to enable the modeling of open-set noise distribution. The estimated SimT is utilized to correct noise issues in pseudo labels and promote the generalization ability of segmentation model on target domain data. In the task of novel target recognition, we first propose closed-to-open label correction (C2OLC) to explicitly derive the supervision signal for open-set classes by exploiting the estimated SimT, and then advance a semantic relation (SR) loss that harnesses the inter-class relation to facilitate the open-set class sample recognition in target domain. Extensive experimental results demonstrate that the proposed SimT can be flexibly plugged into existing DA methods to boost both closed-set and open-set class performance. The source code is available at https://github.com/CityU-AIM-Group/SimT

March on Data Imperfections: Domain Division and Domain Generalization for Semantic Segmentation.

Generalizable model-agnostic semantic segmentation via target-specific normalization

DiGA: Distil to Generalize and then Adapt for Domain Adaptive Semantic Segmentation

Harmonizing Output Imbalance for semantic segmentation on extremely-imbalanced input data

Visual representations with texts domain generalization for semantic segmentation

Threshold-adaptive Unsupervised Focal Loss for Domain Adaptation of Semantic Segmentation

DILRS: Domain-Incremental Learning for Semantic Segmentation in Multi-Source Remote Sensing Data

Domain Adaptive and Generalizable Network Architectures and Training Strategies for Semantic Image Segmentation

Generalizable Medical Image Segmentation via Random Amplitude Mixup and Domain-Specific Image Restoration

Meta-Learned Feature Critics for Domain Generalized Semantic Segmentation

A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation

A Domain Adaptive Semantic Segmentation Method Using Contrastive Learning and Data Augmentation

A Multilevel-Guided Curriculum Domain Adaptation Approach to Semantic Segmentation for High-Resolution Remote Sensing Images.

Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation

Handling Open-set Noise and Novel Target Recognition in Domain Adaptive Semantic Segmentation

Inter-Class and Inter-Domain Semantic Augmentation for Domain Generalization

Augmentation-based Domain Generalization for Semantic Segmentation

Pixel-Level Domain Adaptation: A New Perspective for Enhancing Weakly Supervised Semantic Segmentation

Neutral Cross-Entropy Loss Based Unsupervised Domain Adaptation for Semantic Segmentation

Unsupervised cross domain semantic segmentation with mutual refinement and information distillation

Domain-Invariant Information Aggregation for Domain Generalization Semantic Segmentation