Abstract:Deep learning-based solutions have achieved impressive performance in semantic segmentation but often require large amounts of training data with fine-grained annotations. To alleviate such requisition, a variety of weakly supervised annotation strategies have been proposed, among which scribble supervision is emerging as a popular one due to its user-friendly annotation way. However, the sparsity and diversity of scribble annotations make it nontrivial to train a network to produce deterministic and consistent predictions directly. To address these issues, in this paper we propose holistic solutions involving the design of network structure, loss and training procedure, named CC4S to improve Certainty and Consistency for Scribble-Supervised Semantic Segmentation. Specifically, to reduce uncertainty, CC4S embeds a random walk module into the network structure to make neural representations uniformly distributed within similar semantic regions, which works together with a soft entropy loss function to force the network to produce deterministic predictions. To encourage consistency, CC4S adopts self-supervision training and imposes the consistency loss on the eigenspace of the probability transition matrix in the random walk module (we named neural eigenspace). Such self-supervision inherits the category-level discriminability from the neural eigenspace and meanwhile helps the network focus on producing consistent predictions for the salient parts and neglect semantically heterogeneous backgrounds. Finally, to further improve the performance, CC4S uses the network predictions as pseudo-labels and retrains the network with an extra color constraint regularizer on pseudo-labels to boost semantic consistency in color space. Rich experiments demonstrate the excellent performance of CC4S. In particular, under scribble supervision, CC4S achieves comparable performance to those from fully supervised methods. Comprehensive ablation experiments verify the effectiveness of the design choices in CC4S and its robustness under extreme supervision cases, i.e., when scribbles are shrunk proportionally or dropped randomly. The code for this work has been open-sourced at https://github.com/panzhiyi/CC4S

Scribble-Supervised Semantic Segmentation Inference

Scribble-Supervised Segmentation of Aerial Building Footprints Using Adversarial Learning

Boundary Perception Guidance: A Scribble-Supervised Semantic Segmentation Approach.

Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised Semantic Segmentation with Its Class Label

ZScribbleSeg: Zen and the Art of Scribble Supervised Medical Image Segmentation

BLPSeg: Balance the Label Preference in Scribble-Supervised Semantic Segmentation

CC4S: Encouraging Certainty and Consistency in Scribble-Supervised Semantic Segmentation

Scribble-Supervised Semantic Segmentation by Uncertainty Reduction on Neural Representation and Self-Supervision on Neural Eigenspace.

Scribble-Supervised Semantic Segmentation by Random Walk on Neural Representation and Self-Supervision on Neural Eigenspace

ScribbleSeg: Scribble-based Interactive Image Segmentation

Scribble-Based 3D Shape Segmentation via Weakly-Supervised Learning

Scribble-Based Medical Image Segmentation Via Shape Perturbation Consistency and Boundary Enhancement Constraint

Exploratory Inference Learning for Scribble Supervised Semantic Segmentation.

Improving Semantic Image Segmentation with a Probabilistic Superpixel-Based Dense Conditional Random Field.

PSSD-Transformer: Powerful Sparse Spike-Driven Transformer for Image Semantic Segmentation

ScribbleVS: Scribble-Supervised Medical Image Segmentation via Dynamic Competitive Pseudo Label Selection

ScribbleGen: Generative Data Augmentation Improves Scribble-supervised Semantic Segmentation

Weakly Supervised Medical Image Segmentation Via Superpixel-Guided Scribble Walking and Class-Wise Contrastive Regularization.

Scribble-Supervised Medical Image Segmentation Via Dual-Branch Network and Dynamically Mixed Pseudo Labels Supervision

Weakly‐supervised structural component segmentation via scribble annotations

Scribbles for All: Benchmarking Scribble Supervised Segmentation Across Datasets