Abstract:Weakly supervised learning, releasing deep learning from highly labor-intensive pixel-wise annotations, has gained great attention, especially for medical image segmentation. With only image-level labels, pixel-wise segmentation/localization usually is achieved based on class activation maps (CAMs) containing the most dis-criminative regions. One common consequence of CAM-based approaches is incomplete foreground segmentation, i.e. under-segmentation/false negatives. Meanwhile, suffering from relatively limited medical imaging data, class-irrelevant tissues can hardly be suppressed during classification, resulting in incorrect background identification, i.e. over-segmentation/false positives. The above two issues are determined by the loose-constraint nature of image-level labels penalizing on the entire image space, and thus how to develop pixel-wise constraints based on image-level labels is the key for performance improvement which is under-explored. In this paper, based on unsupervised clustering, we propose a new paradigm called cluster-re-supervision to evaluate the contribution of each pixel in CAMs to final classification and thus generate pixel-wise supervision (i.e., clustering maps) for CAMs refinement on both over- and under-segmentation reduction. Further-more, based on self-supervised learning, an inter-modality image reconstruction module, together with random masking, is designed to complement local information in feature learning which helps stabilize clustering. Experimental results on two popular public datasets demonstrate the superior performance of the proposed weakly-supervised frame-work for medical image segmentation. More importantly, cluster-re-supervision is independent of specific tasks and highly extendable to other applications. The source code of this paper is publicly available at https://github.com/ HustAlexander/Cluster-Re-Supervision.

Learnable Pixel Clustering Via Structure and Semantic Dual Constraints for Unsupervised Image Segmentation.

Pixel-level clustering network for unsupervised image segmentation

Enhancing Unsupervised Semantic Segmentation Through Context-Aware Clustering

Imbalance-Aware Discriminative Clustering for Unsupervised Semantic Segmentation

PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering

Spatial Structure Constraints for Weakly Supervised Semantic Segmentation

Joint learning framework of superpixel generation and fuzzy sparse subspace clustering for color image segmentation

A Lightweight Clustering Framework for Unsupervised Semantic Segmentation

Learning structure-aware semantic segmentation with image-level supervision

Fully Self-Supervised Learning for Semantic Segmentation

Representative Discovery of Structure Cues for Weakly-Supervised Image Segmentation

Diving Segmentation Model into Pixels

End-to-end Boundary Exploration for Weakly-supervised Semantic Segmentation.

Weakly-supervised Semantic Segmentation via Dual-stream Contrastive Learning of Cross-image Contextual Information

3D Shape Segmentation Using Soft Density Peak Clustering and Semi-Supervised Learning

Cluster-Re-Supervision: Bridging the Gap Between Image-Level and Pixel-Wise Labels for Weakly Supervised Medical Image Segmentation

CLUSTSEG: Clustering for Universal Segmentation

Improving Semi-Supervised Semantic Segmentation with Dual-Level Siamese Structure Network

Unsupervised Universal Image Segmentation

Automatic Fuzzy Clustering Framework for Image Segmentation