Abstract:This article studies the problem of learning weakly supervised semantic segmentation (WSSS) from image-level supervision only. Current popular solutions leverage object localization maps from classifiers as supervision for semantic segmentation learning, and struggle to make the localization maps capture more complete object content. Rather than previous efforts that primarily focus on intra-image information, we address the value of cross-image semantic relations for comprehensive object pattern mining. To achieve this, two neural co-attentions are incorporated into the classifier to complementarily capture cross-image semantic similarities and differences. In particular, given a pair of training images, one co-attention enforces the classifier to recognize the common semantics from co-attentive objects, while the other one, called contrastive co-attention, drives the classifier to identify the unique semantics from the rest, unshared objects. This helps the classifier discover more object patterns and better ground semantics in image regions. In addition to boosting object pattern learning, the co-attention can leverage context from other related images to improve localization map inference, hence eventually benefiting semantic segmentation learning. More importantly, our algorithm provides a unified framework that handles well different WSSS settings, i.e., learning WSSS with 1) precise image-level supervision only, 2) extra simple single-label data, and 3) extra noisy web data. Without bells and whistles, it sets new state-of-the-arts on all these settings. Moreover, our approach ranked 1st place in the Weakly-Supervised Semantic Segmentation Track of CVPR2020 Learning from Imperfect Data Challenge. The extensive experimental results demonstrate well the efficacy and high utility of our method.

All-pairs Consistency Learning for Weakly Supervised Semantic Segmentation

SemiCVT: Semi-Supervised Convolutional Vision Transformer for Semantic Segmentation

Dual-Augmented Transformer Network for Weakly Supervised Semantic Segmentation

Semantic-Constraint Matching Transformer for Weakly Supervised Object Localization

Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformers

Semantic-Constraint Matching for Transformer-Based Weakly Supervised Object Localization

Dual Progressive Transformations for Weakly Supervised Semantic Segmentation

L2A: Learning Affinity from Attention for Weakly Supervised Continual Semantic Segmentation

MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentation

Contrastive Tokens and Label Activation for Remote Sensing Weakly Supervised Semantic Segmentation

Looking Beyond Single Images for Weakly Supervised Semantic Segmentation Learning.

APC: Adaptive Patch Contrast for Weakly Supervised Semantic Segmentation

Coupling Global Context and Local Contents for Weakly-Supervised Semantic Segmentation

EGSA: Enhanced and Global Semantic Activation for Weakly Supervised Object Localization.

PCSformer: Pair-wise Cross-scale Sub-prototypes Mining with CNN-transformers for Weakly Supervised Semantic Segmentation

TSD-CAM: transformer-based self distillation with CAM similarity for weakly supervised semantic segmentation

LCTR: On Awakening the Local Continuity of Transformer for Weakly Supervised Object Localization

Activation Modulation and Recalibration Scheme for Weakly Supervised Semantic Segmentation

Attention-based Class Activation Diffusion for Weakly-Supervised Semantic Segmentation

Transformer-CNN Cohort: Semi-supervised Semantic Segmentation by the Best of Both Students

Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation