Abstract:Recently, semi-supervised semantic segmentation methods based on weak-to-strong consistency learning have achieved the most advanced performance. The key to such a technique lies in strong perturbations and multi-objective co-training. However, CutMix, the most commonly used data augmentation in this field, limits the strength of perturbations as it only focuses on single random local context. Besides, complex optimization targets also reduce computational efficiency. In this work, we propose an efficient consistency learning based framework. Specifically, a novel unsupervised data augmentation strategy, EntropyMix, is present for semi-supervised semantic segmentation. Patches of unlabeled data from multi-view augmentations are combined into new training samples based on their prediction entropy, which provides more informative and powerful perturbations for consistency regularization and impels the model to focus on cross-view local context. On this basis, we further propose Self Pseudo Entropy Knowledge Distillation (SPEED) to learn global pixel relations from multi- and cross-view perturbations by optimizing a linear combination of feature-and logit-level distillation loss, enhancing model performance without additional auxiliary segmentation heads or a complex pre-trained teacher model. The collocation of the two ideas above is a plug-and-play technique without additional modification. Extensive experimental results on PASCAL VOC and Cityscapes datasets under various training settings demonstrate the superiority of the proposed data augmentation strategy and self-distillation loss, achieving new state-of-the-art performance. Remarkably, our method reaches mIoU of 75.16% using only 0.87% labeled data on PASCAL VOC and mIoU of 76.98% using only 6.25% labeled data on Cityscapes. The code is available at https://github.com/xiaoqiang-lu/SPEED.

ErrorAug: Making Errors to Find Errors in Semantic Segmentation

Troubleshooting image segmentation models with human-in-the-loop

Robustifying Semantic Cognition of Traversability Across Wearable RGB-depth Cameras

Erroneous Pixel Prediction for Semantic Image Segmentation

Automatic Discovery and Assessment of Interpretable Systematic Errors in Semantic Segmentation

Mitigating Distributional Shift in Semantic Segmentation via Uncertainty Estimation From Unlabeled Data

Mitigating Distributional Shift in Semantic Segmentation via Uncertainty Estimation from Unlabelled Data

Anomaly-Aware Semantic Segmentation by Leveraging Synthetic-Unknown Data

Diagnostics in Semantic Segmentation

Data Augmentation for Semantic Segmentation Using a Real Image Dataset Captured Around the Tsukuba City Hall

Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation

Automated Detection of Label Errors in Semantic Segmentation Datasets via Deep Learning and Uncertainty Quantification

Trusting Semantic Segmentation Networks

Sensitivity-Informed Augmentation for Robust Segmentation

Increasing the Robustness of Semantic Segmentation Models with Painting-by-Numbers

Reliability in Semantic Segmentation: Are We on the Right Track?

Data Augmentation with Illumination Correction in Sematic Segmentation

March on Data Imperfections: Domain Division and Domain Generalization for Semantic Segmentation.

Reliability in Semantic Segmentation: Can We Use Synthetic Data?

Video Semantic Segmentation With Distortion-Aware Feature Correction

Self Pseudo Entropy Knowledge Distillation for Semi-supervised Semantic Segmentation