Abstract:Weakly supervised semantic segmentation (WSSS) with image-level labels intends to achieve dense tasks without laborious annotations. However, due to the ambiguous contexts and fuzzy regions, the performance of WSSS, especially the stages of generating Class Activation Maps (CAMs) and refining pseudo masks, widely suffers from ambiguity while being barely noticed by previous literature. In this work, we propose UniA, a unified single-staged WSSS framework, to efficiently tackle this issue from the perspective of uncertainty inference and affinity diversification, respectively. When activating class objects, we argue that the false activation stems from the bias to the ambiguous regions during the feature extraction. Therefore, we design a more robust feature representation with a probabilistic Gaussian distribution and introduce the uncertainty estimation to avoid the bias. A distribution loss is particularly proposed to supervise the process, which effectively captures the ambiguity and models the complex dependencies among features. When refining pseudo labels, we observe that the affinity from the prevailing refinement methods intends to be similar among ambiguities. To this end, an affinity diversification module is proposed to promote diversity among semantics. A mutual complementing refinement is proposed to initially rectify the ambiguous affinity with multiple inferred pseudo labels. More importantly, a contrastive affinity loss is further designed to diversify the relations among unrelated semantics, which reliably propagates the diversity into the whole feature representations and helps generate better pseudo masks. Extensive experiments are conducted on PASCAL VOC, MS COCO, and medical ACDC datasets, which validate the efficiency of UniA tackling ambiguity and the superiority over recent single-staged or even most multi-staged competitors.

Adaptive Affinity Loss and Erroneous Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation

RWMS: Reliable Weighted Multi-Phase for Semi-supervised Segmentation

Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning

L2A: Learning Affinity from Attention for Weakly Supervised Continual Semantic Segmentation

Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation

Weakly supervised semantic segmentation based on superpixel affinity

Collaborative Semantic Segmentation with Image Labels.

Tackling Ambiguity from Perspective of Uncertainty Inference and Affinity Diversification for Weakly Supervised Semantic Segmentation

Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformers

Progressive Feature Self-reinforcement for Weakly Supervised Semantic Segmentation

Label-efficient Segmentation via Affinity Propagation

A Componentwise Approach to Weakly Supervised Semantic Segmentation Using Dual-Feedback Network

Class-Specific Affinity based Weakly Supervised Semantic Segmentation with Neutral Region Exploration

Modeling the Label Distributions for Weakly-Supervised Semantic Segmentation

Learning Pseudo Labels for Semi-and-weakly Supervised Semantic Segmentation

Weakly Supervised Fine-Grained Semantic Segmentation Via Spatial Correlation-Guided Learning

Weakly-supervised Semantic Segmentation Via Online Pseudo-Mask Correcting

Adaptive Spatial-BCE Loss for Weakly Supervised Semantic Segmentation.

Enhanced Pseudo-Label Generation with Self-supervised Training for Weakly-supervised Semantic Segmentation

Enhancing Weakly Supervised Semantic Segmentation with Multi-modal Foundation Models: An End-to-End Approach