Abstract:This paper studies a practical domain adaptive (DA) semantic segmentation problem where only pseudo-labeled target data is accessible through a black-box model. Due to the domain gap and label shift between two domains, pseudo-labeled target data contains mixed closed-set and open-set label noises. In this paper, we propose a simplex noise transition matrix (SimT) to model the mixed noise distributions in DA semantic segmentation, and leverage SimT to handle open-set label noise and enable novel target recognition. When handling open-set noises, we formulate the problem as estimation of SimT. By exploiting computational geometry analysis and properties of segmentation, we design four complementary regularizers, i.e. volume regularization, anchor guidance, convex guarantee, and semantic constraint, to approximate the true SimT. Specifically, volume regularization minimizes the volume of simplex formed by rows of the non-square SimT, ensuring outputs of model to fit into the ground truth label distribution. To compensate for the lack of open-set knowledge, anchor guidance, convex guarantee, and semantic constraint are devised to enable the modeling of open-set noise distribution. The estimated SimT is utilized to correct noise issues in pseudo labels and promote the generalization ability of segmentation model on target domain data. In the task of novel target recognition, we first propose closed-to-open label correction (C2OLC) to explicitly derive the supervision signal for open-set classes by exploiting the estimated SimT, and then advance a semantic relation (SR) loss that harnesses the inter-class relation to facilitate the open-set class sample recognition in target domain. Extensive experimental results demonstrate that the proposed SimT can be flexibly plugged into existing DA methods to boost both closed-set and open-set class performance. The source code is available at https://github.com/CityU-AIM-Group/SimT

Handling Open-set Noise and Novel Target Recognition in Domain Adaptive Semantic Segmentation

When Masked Image Modeling Meets Source-free Unsupervised Domain Adaptation: Dual-Level Masked Network for Semantic Segmentation

Pulling Target to Source: A New Perspective on Domain Adaptive Semantic Segmentation

Classification Constrained Discriminator for Domain Adaptive Semantic Segmentation

Semantic-Aware Domain Generalized Segmentation.

Unsupervised Domain Adaptive Point Cloud Semantic Segmentation.

March on Data Imperfections: Domain Division and Domain Generalization for Semantic Segmentation.

Unsupervised Domain Adaptive Image Semantic Segmentation Based on Convolutional Fine-Grained Discriminant and Entropy Minimization

Generalizable model-agnostic semantic segmentation via target-specific normalization

Partial Domain Adaptation on Semantic Segmentation

A Multilevel-Guided Curriculum Domain Adaptation Approach to Semantic Segmentation for High-Resolution Remote Sensing Images.

Domain Adaptive Semantic Segmentation Without Source Data

Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation

DOCNet: Dual-Domain Optimized Class-Aware Network for Remote Sensing Image Segmentation

Pixel-Level Domain Adaptation: A New Perspective for Enhancing Weakly Supervised Semantic Segmentation

A Curriculum-style Self-training Approach for Source-Free Semantic Segmentation

Domain Adaptation for Remote Sensing Image Semantic Segmentation: An Integrated Approach of Contrastive Learning and Adversarial Learning

Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation

Multi-source Domain Adaptation for Semantic Segmentation

A Prototypical Metric Learning Approach for Open-Set Semantic Segmentation on Remote Sensing Images

Threshold-adaptive Unsupervised Focal Loss for Domain Adaptation of Semantic Segmentation