Abstract:Deep learning-based solutions have achieved impressive performance in semantic segmentation but often require large amounts of training data with fine-grained annotations. To alleviate such requisition, a variety of weakly supervised annotation strategies have been proposed, among which scribble supervision is emerging as a popular one due to its user-friendly annotation way. However, the sparsity and diversity of scribble annotations make it nontrivial to train a network to produce deterministic and consistent predictions directly. To address these issues, in this paper we propose holistic solutions involving the design of network structure, loss and training procedure, named CC4S to improve Certainty and Consistency for Scribble-Supervised Semantic Segmentation. Specifically, to reduce uncertainty, CC4S embeds a random walk module into the network structure to make neural representations uniformly distributed within similar semantic regions, which works together with a soft entropy loss function to force the network to produce deterministic predictions. To encourage consistency, CC4S adopts self-supervision training and imposes the consistency loss on the eigenspace of the probability transition matrix in the random walk module (we named neural eigenspace). Such self-supervision inherits the category-level discriminability from the neural eigenspace and meanwhile helps the network focus on producing consistent predictions for the salient parts and neglect semantically heterogeneous backgrounds. Finally, to further improve the performance, CC4S uses the network predictions as pseudo-labels and retrains the network with an extra color constraint regularizer on pseudo-labels to boost semantic consistency in color space. Rich experiments demonstrate the excellent performance of CC4S. In particular, under scribble supervision, CC4S achieves comparable performance to those from fully supervised methods. Comprehensive ablation experiments verify the effectiveness of the design choices in CC4S and its robustness under extreme supervision cases, i.e., when scribbles are shrunk proportionally or dropped randomly. The code for this work has been open-sourced at https://github.com/panzhiyi/CC4S

Scribble-Supervised Segmentation of Aerial Building Footprints Using Adversarial Learning

Scribble-Based 3D Shape Segmentation via Weakly-Supervised Learning

Weakly‐supervised structural component segmentation via scribble annotations

ZScribbleSeg: Zen and the Art of Scribble Supervised Medical Image Segmentation

Weakly Supervised Medical Image Segmentation Via Superpixel-Guided Scribble Walking and Class-Wise Contrastive Regularization.

Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised Semantic Segmentation with Its Class Label

Scribble-Supervised Semantic Segmentation by Uncertainty Reduction on Neural Representation and Self-Supervision on Neural Eigenspace.

Embarrassingly Simple Scribble Supervision for 3D Medical Segmentation

ScribbleVS: Scribble-Supervised Medical Image Segmentation via Dynamic Competitive Pseudo Label Selection

Scribble-Supervised Semantic Segmentation by Random Walk on Neural Representation and Self-Supervision on Neural Eigenspace

Integrated Framework for Unsupervised Building Segmentation with Segment Anything Model-Based Pseudo-Labeling and Weakly Supervised Learning

CC4S: Encouraging Certainty and Consistency in Scribble-Supervised Semantic Segmentation

Expediting Building Footprint Extraction from High-resolution Remote Sensing Images via progressive lenient supervision

ScribSD+: Scribble-supervised medical image segmentation based on simultaneous multi-scale knowledge distillation and class-wise contrastive regularization

Building Extraction From High-Resolution Aerial Imagery Using A Generative Adversarial Network With Spatial And Channel Attention Mechanisms

Optimal Scale of Hierarchical Image Segmentation with Scribbles Guidance for Weakly Supervised Semantic Segmentation

A Semantic Segmentation Network for Urban-Scale Building Footprint Extraction Using RGB Satellite Imagery

An Unsupervised Domain Adaption Framework for Aerial Image Semantic Segmentation Based on Curriculum Learning

Scribbles for All: Benchmarking Scribble Supervised Segmentation Across Datasets

Boundary Perception Guidance: A Scribble-Supervised Semantic Segmentation Approach.

Structure-Aware Weakly Supervised Network for Building Extraction from Remote Sensing Images.