Abstract:Weakly-Supervised Semantic Segmentation (WSSS) aims to train segmentation models by weak labels, which is receiving significant attention due to its low annotation cost. Existing approaches focus on generating pseudo labels for supervision while largely ignoring to leverage the inherent semantic correlation among different pseudo labels. We observe that pseudo-labeled pixels that are close to each other in the feature space are more likely to share the same class, and those closer to the distribution centers tend to have higher confidence. Motivated by this, we propose to model the underlying label distributions and employ cross-label constraints to generate more accurate pseudo labels. In this paper, we develop a unified WSSS framework named Adaptive Gaussian Mixtures Model, which leverages a GMM to model the label distributions. Specifically, we calculate the feature distribution centers of pseudo-labeled pixels and build the GMM by measuring the distance between the centers and each pseudo-labeled pixel. Then, we introduce an Online Expectation-Maximization (OEM) algorithm and a novel maximization loss to optimize the GMM adaptively, aiming to learn more discriminative decision boundaries between different class-wise Gaussian mixtures. Based on the label distributions, we leverage the GMM to generate high-quality pseudo labels for more reliable supervision. Our framework is capable of solving different forms of weak labels: image-level labels, points, scribbles, blocks, and bounding-boxes. Extensive experiments on PASCAL, COCO, Cityscapes, and ADE20K datasets demonstrate that our framework can effectively provide more reliable supervision and outperform the state-of-the-art methods under all settings. Code will be available at

Learning from Weak and Noisy Labels for Semantic Segmentation.

Learning from weak and noisy labels for semantic segmentation

Learning Effectively from Noisy Supervision for Weakly Supervised Semantic Segmentation.

Weakly Supervised Remote Sensing Image Semantic Segmentation with Pseudo Label Noise Suppression

Large-Scale Sparse Learning from Noisy Tags for Semantic Segmentation.

Large-scale sparse learning from noisy tags for semantic segmentation

Modeling the Label Distributions for Weakly-Supervised Semantic Segmentation

Learning from Pixel-Level Label Noise: A New Perspective for Semi-Supervised Semantic Segmentation.

Sparse Reconstruction for Weakly Supervised Semantic Segmentation.

W2P: Switching from Weak Supervision to Partial Supervision for Semantic Segmentation

Weakly-supervised Semantic Segmentation Via Online Pseudo-Mask Correcting

Weakly-Supervised Semantic Segmentation with Image-Level Labels: from Traditional Models to Foundation Models

Weakly Supervised Semantic Segmentation for Social Images

Learning Pseudo Labels for Semi-and-weakly Supervised Semantic Segmentation

Superpixel Consistency Saliency Map Generation for Weakly Supervised Semantic Segmentation of Remote Sensing Images

Learning Visual Words for Weakly-Supervised Semantic Segmentation

Weakly supervised semantic segmentation based on superpixel affinity

Adaptive Affinity Loss and Erroneous Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation

A Weakly Supervised Semantic Segmentation Method Based on Local Superpixel Transformation

Mining Pixels: Weakly Supervised Semantic Segmentation Using Image Labels

Hybrid Supervised Instance Segmentation by Learning Label Noise Suppression