Abstract:Weakly supervised semantic segmentation (WSSS) with image-level labels intends to achieve dense tasks without laborious annotations. However, due to the ambiguous contexts and fuzzy regions, the performance of WSSS, especially the stages of generating Class Activation Maps (CAMs) and refining pseudo masks, widely suffers from ambiguity while being barely noticed by previous literature. In this work, we propose UniA, a unified single-staged WSSS framework, to efficiently tackle this issue from the perspective of uncertainty inference and affinity diversification, respectively. When activating class objects, we argue that the false activation stems from the bias to the ambiguous regions during the feature extraction. Therefore, we design a more robust feature representation with a probabilistic Gaussian distribution and introduce the uncertainty estimation to avoid the bias. A distribution loss is particularly proposed to supervise the process, which effectively captures the ambiguity and models the complex dependencies among features. When refining pseudo labels, we observe that the affinity from the prevailing refinement methods intends to be similar among ambiguities. To this end, an affinity diversification module is proposed to promote diversity among semantics. A mutual complementing refinement is proposed to initially rectify the ambiguous affinity with multiple inferred pseudo labels. More importantly, a contrastive affinity loss is further designed to diversify the relations among unrelated semantics, which reliably propagates the diversity into the whole feature representations and helps generate better pseudo masks. Extensive experiments are conducted on PASCAL VOC, MS COCO, and medical ACDC datasets, which validate the efficiency of UniA tackling ambiguity and the superiority over recent single-staged or even most multi-staged competitors.

UAFer: A Unified Model for Class-agnostic Binary Segmentation with Uncertainty-Aware Feature Reassembly

An Image Segmentation Method Based on Transformer and Multi-Scale Feature Fusion for UAV Marine Environment Monitoring

Unified Feature Consistency of Under-Performing Pixels and Valid Regions for Semi-Supervised Medical Image Segmentation

Tackling Ambiguity from Perspective of Uncertainty Inference and Affinity Diversification for Weakly Supervised Semantic Segmentation

Uncertainty-aware Cascade Network for Ultrasound Image Segmentation with Ambiguous Boundary

An Augmentation-based Model Re-adaptation Framework for Robust Image Segmentation

Learning to segment subcortical structures from noisy annotations with a novel uncertainty-reliability aware learning framework

Elongated Physiological Structure Segmentation via Spatial and Scale Uncertainty-aware Network

PLUG: Revisiting Amodal Segmentation with Foundation Model and Hierarchical Focus

Video Object Segmentation via Structural Feature Reconfiguration

Uncertainty-Aware Source-Free Domain Adaptive Semantic Segmentation.

A Unified Efficient Pyramid Transformer for Semantic Segmentation

AF$_2$: Adaptive Focus Framework for Aerial Imagery Segmentation

Attention-Guided Multi-Scale Fusion Network for Similar Objects Semantic Segmentation

UDNet: Uncertainty-aware Deep Network for Salient Object Detection

Hyperbolic Uncertainty Aware Semantic Segmentation

Uncertainty-aware domain alignment for anatomical structure segmentation

Uncertainty Quantification in Medical Image Segmentation with Multi-decoder U-Net

SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentation

BFANet: Effective Segmentation Network for Low Altitude High-Resolution Urban Scene Image

Unsupervised segmentation via semantic-apparent feature fusion