Abstract:Most advanced weakly supervised semantic segmentation studies have explored various strategies for refining the class activation mapping (CAM) maps and thus producing pixelwise pseudomasks for training segmentation networks. However, CAM-based approaches suffer from the limited receptive fields of convolutional neural networks (CNNs) when exploring long-range feature dependencies, resulting in incomplete object region detection and inferior segmentation performance. On the other hand, the self-attention mechan ism has proven to be effective in modeling global contexts, providing a potential perspective for producing integral activation maps. Nevertheless, the use of self-attention maps to generate pseudomasks faces significant obstacles, as the attention activations are class-agnostic, locally inconsistent and noisy. To address these issues, we propose a novel end-to-end transformer-based framework, namely the g radient-coupled c ross- p atch a ttention m ap (GC-PAM) . First, the classification score gradients are backpropagated to recover the semantic knowledge of the attention map, which is then used to retrieve the object-relevant patches and object-irrelevant patches. Next, the corresponding object patch attention maps are aggregated through content-adaptive cross-patch coupling to construct comprehensive activation maps, whereas the object-irrelevant maps are employed to suppress the background noise. Subsequently, the resulting object activation maps are used to generate pseudolabels to supervise the segmentation branch. Extensive experiments are conducted to demonstrate the effectiveness of the proposed GC-PAM. Despite its simplicity and computational efficiency, the GC-PAM with the DeiT-S backbone surpasses the existing state-of-the-art approaches on the PASCAL VOC 2012 benchmark (75.3% val , 74.6% test ). The results also demonstrate that the GC-PAM is a feasible alternative to CNN-based architectures.

Enhanced Online CAM: Single-stage Weakly Supervised Semantic Segmentation Via Collaborative Guidance

Exploit CAM by itself: Complementary Learning System for Weakly Supervised Semantic Segmentation

Exploit CAM by Itself: Complementary Learning System for Weakly Supervised Semantic Segmentation

C-CAM: Causal CAM for Weakly Supervised Semantic Segmentation on Medical Image

CFCG: Semi-Supervised Semantic Segmentation Via Cross-Fusion and Contour Guidance Supervision

Rethinking CAM in Weakly-Supervised Semantic Segmentation

Weakly Supervised Co-training with Swapping Assignments for Semantic Segmentation

DCAM: Disturbed Class Activation Maps for Weakly Supervised Semantic Segmentation

Self-Supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation

CG-fusion CAM: Online segmentation of laser-induced damage on large-aperture optics

Weakly Supervised Semantic Segmentation via Progressive Patch Learning

Gradient-coupled Cross-Patch Attention Map for Weakly Supervised Semantic Segmentation

SFC: Shared Feature Calibration in Weakly Supervised Semantic Segmentation

TSD-CAM: transformer-based self distillation with CAM similarity for weakly supervised semantic segmentation

Multi-Granularity Denoising and Bidirectional Alignment for Weakly Supervised Semantic Segmentation

MS-CAM: Multi-Scale Class Activation Maps for Weakly-Supervised Segmentation of Geographic Atrophy Lesions in SD-OCT Images

Weakly Supervised Fine-Grained Semantic Segmentation Via Spatial Correlation-Guided Learning

Erase then Grow: Generating Correct Class Activation Maps for Weakly-Supervised Semantic Segmentation

Group-CAM: Group Score-Weighted Visual Explanations for Deep Convolutional Networks

Enhanced Pseudo-Label Generation with Self-supervised Training for Weakly-supervised Semantic Segmentation

Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast