Abstract:Obtaining image-level class labels for remote sensing (RS) images is a relatively straightforward process, sparking significant interest in weakly supervised semantic segmentation (WSSS). However, RS images present challenges beyond those encountered in generic WSSS, including complex backgrounds, densely distributed small objects, and considerable scale variations. To address the above issues, we introduce a consistency-constrained multiclass attention model, noted as CocoaNet. Specifically, CocoaNet endeavors to capture both semantic correlation and class distinctiveness using a global-local adaptive attention mechanism, which integrates the self-attention to model global correlation, complemented by a local perception branch that intensifies focus on local regions. The resulting class-specific attention weights and the patch-level pairwise affinity weights are employed to optimize the initial class activation maps (CAMs). This mechanism proves highly effective in mitigating interclass interference and managing the distribution of densely clustered small objects. Moreover, we invoke a consistency constraint to rectify activation inaccuracy. By utilizing a Siamese structure for the mutual supervision of features extracted from images at different scales, we address substantial scale variations in RS scenes. Simultaneously, a class contrast loss is adopted to enhance the discriminativeness of class-specific features. Departing from the conventional CAM optimization, which is rather complex and time-consuming, we harness the prior knowledge from the generic segment anything model (SAM) to design a joint optimization strategy (JOS) that refines target boundaries and further promotes discriminative visual features. We validate the effectiveness of our proposed approach on three benchmark datasets in multiclass RS scenarios, and the experimental results demonstrate that our model yields promising advancements compared to state-of-the-art methods.

Negative Class Guided Spatial Consistency Network for Sparsely Supervised Semantic Segmentation of Remote Sensing Images

Optimizing Spatial Relationships in GCN to Improve the Classification Accuracy of Remote Sensing Images

Edge-Enhanced GCIFFNet: A Multiclass Semantic Segmentation Network Based on Edge Enhancement and Multiscale Attention Mechanism

Global Context Dependencies Aware Network for Efficient Semantic Segmentation of Fine-Resolution Remoted Sensing Images

SSCNet: A Spectrum-Space Collaborative Network for Semantic Segmentation of Remote Sensing Images

A Spectral–Spatial Context-Boosted Network for Semantic Segmentation of Remote Sensing Images

Semi-Supervised Semantic Segmentation of Remote Sensing Images With Iterative Contrastive Network

CGGLNet: Semantic Segmentation Network for Remote Sensing Images Based on Category-Guided Global–Local Feature Interaction

Self-constructing graph neural networks to model long-range pixel dependencies for semantic segmentation of remote sensing images

SEMI-SUPERVISED SEMANTIC SEGMENTATION NETWORK VIA LEARNING CONSISTENCY FOR REMOTE SENSING LAND-COVER CLASSIFICATION

Scale-Aware Neural Network for Semantic Segmentation of Multi-Resolution Remote Sensing Images

Semantic Consistency Networks for 3D Object Detection.

Densely Based Multi-Scale and Multi-Modal Fully Convolutional Networks for High-Resolution Remote-Sensing Image Semantic Segmentation

Semantic segmentation for remote sensing images via dense feature extraction and companion loss neural network

UGCNet: An Unsupervised Semantic Segmentation Network Embedded With Geometry Consistency for Remote-Sensing Images

Spatial and Semantic Consistency Contrastive Learning for Self-Supervised Semantic Segmentation of Remote Sensing Images

Spatial-Temporal Semantic Perception Network for Remote Sensing Image Semantic Change Detection

SE2Net: Semantic Segmentation of Remote Sensing Images Based on Self-Attention and Edge Enhancement Modules

Weakly Supervised Semantic Segmentation With Consistency-Constrained Multiclass Attention for Remote Sensing Scenes

Weakly Supervised Semantic Segmentation with Consistency-Constrained Multi-Class Attention for Remote Sensing Scenes

SCAttNet: Semantic Segmentation Network with Spatial and Channel Attention Mechanism for High-Resolution Remote Sensing Images