Abstract:Obtaining image-level class labels for remote sensing (RS) images is a relatively straightforward process, sparking significant interest in weakly supervised semantic segmentation (WSSS). However, RS images present challenges beyond those encountered in generic WSSS, including complex backgrounds, densely distributed small objects, and considerable scale variations. To address the above issues, we introduce a consistency-constrained multiclass attention model, noted as CocoaNet. Specifically, CocoaNet endeavors to capture both semantic correlation and class distinctiveness using a global-local adaptive attention mechanism, which integrates the self-attention to model global correlation, complemented by a local perception branch that intensifies focus on local regions. The resulting class-specific attention weights and the patch-level pairwise affinity weights are employed to optimize the initial class activation maps (CAMs). This mechanism proves highly effective in mitigating interclass interference and managing the distribution of densely clustered small objects. Moreover, we invoke a consistency constraint to rectify activation inaccuracy. By utilizing a Siamese structure for the mutual supervision of features extracted from images at different scales, we address substantial scale variations in RS scenes. Simultaneously, a class contrast loss is adopted to enhance the discriminativeness of class-specific features. Departing from the conventional CAM optimization, which is rather complex and time-consuming, we harness the prior knowledge from the generic segment anything model (SAM) to design a joint optimization strategy (JOS) that refines target boundaries and further promotes discriminative visual features. We validate the effectiveness of our proposed approach on three benchmark datasets in multiclass RS scenarios, and the experimental results demonstrate that our model yields promising advancements compared to state-of-the-art methods.

Class-Specific Affinity based Weakly Supervised Semantic Segmentation with Neutral Region Exploration

Adaptive Affinity Loss and Erroneous Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation

Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning

Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation

Scale-aware Attention Network for Weakly Supervised Semantic Segmentation

Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentation

Weakly supervised semantic segmentation based on superpixel affinity

Class Semantic Enhancement Network for Semantic Segmentation

Adaptive Activation Network for Weakly Supervised Semantic Segmentation

Weak-shot Semantic Segmentation by Transferring Semantic Affinity and Boundary

Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformers

Attention Based Object Localization for Weakly Supervised Semantic Segmentation

L2A: Learning Affinity from Attention for Weakly Supervised Continual Semantic Segmentation

CIAN: Cross-Image Affinity Net for Weakly Supervised Semantic Segmentation

Learning Hybrid Semantic Affinity for Point Cloud Segmentation

Multi-modality Affinity Inference for Weakly Supervised 3D Semantic Segmentation

Towards Single Stage Weakly Supervised Semantic Segmentation

CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning

Saliency Guided Self-Attention Network for Weakly and Semi-Supervised Semantic Segmentation

Weakly Supervised Semantic Segmentation with Consistency-Constrained Multi-Class Attention for Remote Sensing Scenes