Abstract:This paper focuses on the unsupervised domain adaptation of transferring the knowledge from the source domain to the target domain in the context of semantic segmentation. Existing approaches usually regard the pseudo label as the ground truth to fully exploit the unlabeled target-domain data. Yet the pseudo labels of the target-domain data are usually predicted by the model trained on the source domain. Thus, the generated labels inevitably contain the incorrect prediction due to the discrepancy between the training domain and the test domain, which could be transferred to the final adapted model and largely compromises the training process. To overcome the problem, this paper proposes to explicitly estimate the prediction uncertainty during training to rectify the pseudo label learning for unsupervised semantic segmentation adaptation. Given the input image, the model outputs the semantic segmentation prediction as well as the uncertainty of the prediction. Specifically, we model the uncertainty via the prediction variance and involve the uncertainty into the optimization objective. To verify the effectiveness of the proposed method, we evaluate the proposed method on two prevalent synthetic-to-real semantic segmentation benchmarks, i.e., GTA5 \(\rightarrow \) Cityscapes and SYNTHIA \(\rightarrow \) Cityscapes, as well as one cross-city benchmark, i.e., Cityscapes \(\rightarrow \) Oxford RobotCar. We demonstrate through extensive experiments that the proposed approach (1) dynamically sets different confidence thresholds according to the prediction variance, (2) rectifies the learning from noisy pseudo labels, and (3) achieves significant improvements over the conventional pseudo label learning and yields competitive performance on all three benchmarks.

Pseudo Label Fusion With Uncertainty Estimation for Semi-Supervised Cropping Box Regression

Pseudo-Labeling by Multi-Policy Viewfinder Network for Image Cropping

Weakly Supervised Instance Segmentation Using Multi-Prior Fusion.

Fully Data-Driven Pseudo Label Estimation for Pointly-Supervised Panoptic Segmentation

Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation

Uncertainty-aware pseudo-label and consistency for semi-supervised medical image segmentation

Synergy-Guided Regional Supervision of Pseudo Labels for Semi-Supervised Medical Image Segmentation

Mutually aided uncertainty incorporated Dual Consistency Regularization with Pseudo Label for Semi-Supervised Medical Image Segmentation

Uncertain region mining semi-supervised object detection

Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

Weakly Supervised Region of Interest Extraction Based on Uncertainty-Aware Self-Refinement Learning for Remote Sensing Images

Learning Pseudo Labels for Semi-and-weakly Supervised Semantic Segmentation

Using Unreliable Pseudo-Labels for Label-Efficient Semantic Segmentation

Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression

USD: U ncertainty-based One-phase Learning to Enhance Pseudo-Label Reliability for S emi-Supervised Object D etection

Confidence-aware Pseudo-label Learning for Weakly Supervised Visual Grounding

Semantic Segmentation of Weakly Annotated Remote Sensing Images Based on Feature Adversary and Uncertainty Perception

Self-Loop Uncertainty: A Novel Pseudo-Label for Semi-Supervised Medical Image Segmentation

In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning

Adaptive Affinity Loss and Erroneous Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation