Abstract:Most advanced weakly supervised semantic segmentation studies have explored various strategies for refining the class activation mapping (CAM) maps and thus producing pixelwise pseudomasks for training segmentation networks. However, CAM-based approaches suffer from the limited receptive fields of convolutional neural networks (CNNs) when exploring long-range feature dependencies, resulting in incomplete object region detection and inferior segmentation performance. On the other hand, the self-attention mechan ism has proven to be effective in modeling global contexts, providing a potential perspective for producing integral activation maps. Nevertheless, the use of self-attention maps to generate pseudomasks faces significant obstacles, as the attention activations are class-agnostic, locally inconsistent and noisy. To address these issues, we propose a novel end-to-end transformer-based framework, namely the g radient-coupled c ross- p atch a ttention m ap (GC-PAM) . First, the classification score gradients are backpropagated to recover the semantic knowledge of the attention map, which is then used to retrieve the object-relevant patches and object-irrelevant patches. Next, the corresponding object patch attention maps are aggregated through content-adaptive cross-patch coupling to construct comprehensive activation maps, whereas the object-irrelevant maps are employed to suppress the background noise. Subsequently, the resulting object activation maps are used to generate pseudolabels to supervise the segmentation branch. Extensive experiments are conducted to demonstrate the effectiveness of the proposed GC-PAM. Despite its simplicity and computational efficiency, the GC-PAM with the DeiT-S backbone surpasses the existing state-of-the-art approaches on the PASCAL VOC 2012 benchmark (75.3% val , 74.6% test ). The results also demonstrate that the GC-PAM is a feasible alternative to CNN-based architectures.

Background Noise Reduction of Attention Map for Weakly Supervised Semantic Segmentation

ICTCAM: Introducing Convolution to Transformer-Based Weakly Supervised Semantic Segmentation

Attention-based Class Activation Diffusion for Weakly-Supervised Semantic Segmentation

DCAM: Disturbed Class Activation Maps for Weakly Supervised Semantic Segmentation

Attention Based Object Localization for Weakly Supervised Semantic Segmentation

Rethinking CAM in Weakly-Supervised Semantic Segmentation

TSD-CAM: transformer-based self distillation with CAM similarity for weakly supervised semantic segmentation

Gradient-coupled Cross-Patch Attention Map for Weakly Supervised Semantic Segmentation

Dual Progressive Transformations for Weakly Supervised Semantic Segmentation

Non-target Feature Filtering for Weakly Supervised Semantic Segmentation

Wave-Like Class Activation Map with Representation Fusion for Weakly-Supervised Semantic Segmentation

A Weakly Supervised Semantic Segmentation Method Based on Local Superpixel Transformation

Activation Modulation and Recalibration Scheme for Weakly Supervised Semantic Segmentation

Extracting Class Activation Maps from Non-Discriminative Features as well

Joint Image Restoration for Weakly Supervised Semantic Segmentation

Weakly supervised semantic segmentation based on superpixel affinity

Superpixel Consistency Saliency Map Generation for Weakly Supervised Semantic Segmentation of Remote Sensing Images

Average Activation Network for Weakly Supervised Semantic Segmentation.

Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

Area Intervention for Enhancing Class Activation Maps in Weakly Supervised Semantic Segmentation

Multi-Granularity Denoising and Bidirectional Alignment for Weakly Supervised Semantic Segmentation