Abstract:Deep learning-based solutions have achieved impressive performance in semantic segmentation but often require large amounts of training data with fine-grained annotations. To alleviate such requisition, a variety of weakly supervised annotation strategies have been proposed, among which scribble supervision is emerging as a popular one due to its user-friendly annotation way. However, the sparsity and diversity of scribble annotations make it nontrivial to train a network to produce deterministic and consistent predictions directly. To address these issues, in this paper we propose holistic solutions involving the design of network structure, loss and training procedure, named CC4S to improve Certainty and Consistency for Scribble-Supervised Semantic Segmentation. Specifically, to reduce uncertainty, CC4S embeds a random walk module into the network structure to make neural representations uniformly distributed within similar semantic regions, which works together with a soft entropy loss function to force the network to produce deterministic predictions. To encourage consistency, CC4S adopts self-supervision training and imposes the consistency loss on the eigenspace of the probability transition matrix in the random walk module (we named neural eigenspace). Such self-supervision inherits the category-level discriminability from the neural eigenspace and meanwhile helps the network focus on producing consistent predictions for the salient parts and neglect semantically heterogeneous backgrounds. Finally, to further improve the performance, CC4S uses the network predictions as pseudo-labels and retrains the network with an extra color constraint regularizer on pseudo-labels to boost semantic consistency in color space. Rich experiments demonstrate the excellent performance of CC4S. In particular, under scribble supervision, CC4S achieves comparable performance to those from fully supervised methods. Comprehensive ablation experiments verify the effectiveness of the design choices in CC4S and its robustness under extreme supervision cases, i.e., when scribbles are shrunk proportionally or dropped randomly. The code for this work has been open-sourced at https://github.com/panzhiyi/CC4S

Optimal Scale of Hierarchical Image Segmentation with Scribbles Guidance for Weakly Supervised Semantic Segmentation

Scribble-Supervised Segmentation of Aerial Building Footprints Using Adversarial Learning

Scribble-Based 3D Shape Segmentation via Weakly-Supervised Learning

ZScribbleSeg: Zen and the Art of Scribble Supervised Medical Image Segmentation

Weakly Supervised Medical Image Segmentation Via Superpixel-Guided Scribble Walking and Class-Wise Contrastive Regularization.

CC4S: Encouraging Certainty and Consistency in Scribble-Supervised Semantic Segmentation

Size Aware Cross-shape Scribble Supervision for Medical Image Segmentation

ScribbleVS: Scribble-Supervised Medical Image Segmentation via Dynamic Competitive Pseudo Label Selection

DMSPS: Dynamically mixed soft pseudo-label supervision for scribble-supervised medical image segmentation

Scribble-Supervised Semantic Segmentation by Uncertainty Reduction on Neural Representation and Self-Supervision on Neural Eigenspace.

ScribSD+: Scribble-supervised medical image segmentation based on simultaneous multi-scale knowledge distillation and class-wise contrastive regularization

Efficient Object Region Discovery for Weakly-supervised Semantic Segmentation

Weakly‐supervised structural component segmentation via scribble annotations

Embarrassingly Simple Scribble Supervision for 3D Medical Segmentation

Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised Semantic Segmentation with Its Class Label

Boundary Perception Guidance: A Scribble-Supervised Semantic Segmentation Approach.

Scribble-based 3D Multiple Abdominal Organ Segmentation via Triple-branch Multi-dilated Network with Pixel- and Class-wise Consistency

Scribble-Supervised Semantic Segmentation by Random Walk on Neural Representation and Self-Supervision on Neural Eigenspace

Scribbles for All: Benchmarking Scribble Supervised Segmentation Across Datasets

Coarse-to-Fine Annotation Enrichment for Semantic Segmentation Learning

STC: A Simple to Complex Framework for Weakly-Supervised Semantic Segmentation