Abstract:Most advanced weakly supervised semantic segmentation studies have explored various strategies for refining the class activation mapping (CAM) maps and thus producing pixelwise pseudomasks for training segmentation networks. However, CAM-based approaches suffer from the limited receptive fields of convolutional neural networks (CNNs) when exploring long-range feature dependencies, resulting in incomplete object region detection and inferior segmentation performance. On the other hand, the self-attention mechan ism has proven to be effective in modeling global contexts, providing a potential perspective for producing integral activation maps. Nevertheless, the use of self-attention maps to generate pseudomasks faces significant obstacles, as the attention activations are class-agnostic, locally inconsistent and noisy. To address these issues, we propose a novel end-to-end transformer-based framework, namely the g radient-coupled c ross- p atch a ttention m ap (GC-PAM) . First, the classification score gradients are backpropagated to recover the semantic knowledge of the attention map, which is then used to retrieve the object-relevant patches and object-irrelevant patches. Next, the corresponding object patch attention maps are aggregated through content-adaptive cross-patch coupling to construct comprehensive activation maps, whereas the object-irrelevant maps are employed to suppress the background noise. Subsequently, the resulting object activation maps are used to generate pseudolabels to supervise the segmentation branch. Extensive experiments are conducted to demonstrate the effectiveness of the proposed GC-PAM. Despite its simplicity and computational efficiency, the GC-PAM with the DeiT-S backbone surpasses the existing state-of-the-art approaches on the PASCAL VOC 2012 benchmark (75.3% val , 74.6% test ). The results also demonstrate that the GC-PAM is a feasible alternative to CNN-based architectures.

Cross-Patch Relation Enhanced for Weakly Supervised Semantic Segmentation

Complementary Patch for Weakly Supervised Semantic Segmentation

Weakly Supervised Semantic Segmentation with Patch-Based Metric Learning Enhancement

PatchNet: Maximize the Exploration of Congeneric Semantics for Weakly Supervised Semantic Segmentation

Weakly Supervised Semantic Segmentation via Progressive Patch Learning

Weakly supervised semantic segmentation based on superpixel affinity

Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

Gradient-coupled Cross-Patch Attention Map for Weakly Supervised Semantic Segmentation

Cross-supervision-based equilibrated fusion mechanism of local and global attention for semantic segmentation

ECT: Enhancement Cross Training for Weakly Supervised Semantic Segmentation

Coupling Global Context and Local Contents for Weakly-Supervised Semantic Segmentation

Superpixel Consistency Saliency Map Generation for Weakly Supervised Semantic Segmentation of Remote Sensing Images

APC: Adaptive Patch Contrast for Weakly Supervised Semantic Segmentation

A Weakly Supervised Semantic Segmentation Method Based on Local Superpixel Transformation

Activation extending based on long-range dependencies for weakly supervised semantic segmentation

MoRe: Class Patch Attention Needs Regularization for Weakly Supervised Semantic Segmentation

Attention Based Object Localization for Weakly Supervised Semantic Segmentation

Cross-Pixel Dependency with Boundary-Feature Transformation for Weakly Supervised Semantic Segmentation

Activation Modulation and Recalibration Scheme for Weakly Supervised Semantic Segmentation

Top-K Pooling with Patch Contrastive Learning for Weakly-Supervised Semantic Segmentation

DCAM: Disturbed Class Activation Maps for Weakly Supervised Semantic Segmentation