Abstract:Deep learning-based solutions have achieved impressive performance in semantic segmentation but often require large amounts of training data with fine-grained annotations. To alleviate such requisition, a variety of weakly supervised annotation strategies have been proposed, among which scribble supervision is emerging as a popular one due to its user-friendly annotation way. However, the sparsity and diversity of scribble annotations make it nontrivial to train a network to produce deterministic and consistent predictions directly. To address these issues, in this paper we propose holistic solutions involving the design of network structure, loss and training procedure, named CC4S to improve Certainty and Consistency for Scribble-Supervised Semantic Segmentation. Specifically, to reduce uncertainty, CC4S embeds a random walk module into the network structure to make neural representations uniformly distributed within similar semantic regions, which works together with a soft entropy loss function to force the network to produce deterministic predictions. To encourage consistency, CC4S adopts self-supervision training and imposes the consistency loss on the eigenspace of the probability transition matrix in the random walk module (we named neural eigenspace). Such self-supervision inherits the category-level discriminability from the neural eigenspace and meanwhile helps the network focus on producing consistent predictions for the salient parts and neglect semantically heterogeneous backgrounds. Finally, to further improve the performance, CC4S uses the network predictions as pseudo-labels and retrains the network with an extra color constraint regularizer on pseudo-labels to boost semantic consistency in color space. Rich experiments demonstrate the excellent performance of CC4S. In particular, under scribble supervision, CC4S achieves comparable performance to those from fully supervised methods. Comprehensive ablation experiments verify the effectiveness of the design choices in CC4S and its robustness under extreme supervision cases, i.e., when scribbles are shrunk proportionally or dropped randomly. The code for this work has been open-sourced at https://github.com/panzhiyi/CC4S

Weakly‐supervised structural component segmentation via scribble annotations

Scribble-Supervised Segmentation of Aerial Building Footprints Using Adversarial Learning

Scribble-Based 3D Shape Segmentation via Weakly-Supervised Learning

Weakly Supervised Medical Image Segmentation Via Superpixel-Guided Scribble Walking and Class-Wise Contrastive Regularization.

ZScribbleSeg: Zen and the Art of Scribble Supervised Medical Image Segmentation

CC4S: Encouraging Certainty and Consistency in Scribble-Supervised Semantic Segmentation

Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised Semantic Segmentation with Its Class Label

Embarrassingly Simple Scribble Supervision for 3D Medical Segmentation

Weakly-Supervised Salient Object Detection via Scribble Annotations

ScribbleVS: Scribble-Supervised Medical Image Segmentation via Dynamic Competitive Pseudo Label Selection

Size Aware Cross-shape Scribble Supervision for Medical Image Segmentation

Optimal Scale of Hierarchical Image Segmentation with Scribbles Guidance for Weakly Supervised Semantic Segmentation

ScribFormer: Transformer Makes CNN Work Better for Scribble-based Medical Image Segmentation

Edge-Aware Vessel Segmentation Using Scribble Supervision

Weakly Supervised LiDAR Semantic Segmentation via Scatter Image Annotation

Sribble Supervised Multimodal Medical Image Segmentation

Scribble-Supervised Semantic Segmentation by Uncertainty Reduction on Neural Representation and Self-Supervision on Neural Eigenspace.

Structure-Aware Weakly Supervised Network for Building Extraction from Remote Sensing Images.

ScribSD+: Scribble-supervised medical image segmentation based on simultaneous multi-scale knowledge distillation and class-wise contrastive regularization

Scribble-based 3D Multiple Abdominal Organ Segmentation via Triple-branch Multi-dilated Network with Pixel- and Class-wise Consistency