Abstract:Instance segmentation in remote sensing (RS) imagery aims to predict the locations of instances and represent them with pixel-level masks. Thanks to the more accurate pixel-level information for each instance, instance segmentation has enormous potential applications in resource planning, urban surveillance, and military reconnaissance. However, current RS imagery instance segmentation methods mostly follow the fully supervised paradigm, relying on expensive pixel-level labels. Moreover, remote sensing imagery suffers from cluttered backgrounds and significant variations in target scales, making segmentation challenging. To accommodate these limitations, we propose a semantic attention enhancement and structured model-guided multi-scale weakly supervised instance segmentation network (SASM-Net). Building upon the modeling of spatial relationships for weakly supervised instance segmentation, we further design the multi-scale feature extraction module (MSFE module), semantic attention enhancement module (SAE module), and structured model guidance module (SMG module) for SASM-Net to enable a balance between label production costs and visual processing. The MSFE module adopts a hierarchical approach similar to the residual structure to establish equivalent feature scales and to adapt to the significant scale variations of instances in RS imagery. The SAE module is a dual-stream structure with semantic information prediction and attention enhancement streams. It can enhance the network's activation of instances in the images and reduce cluttered backgrounds' interference. The SMG module can assist the SAE module in the training process to construct supervision with edge information, which can implicitly lead the model to a representation with structured inductive bias, reducing the impact of the low sensitivity of the model to edge information caused by the lack of fine-grained pixel-level labeling. Experimental results indicate that the proposed SASM-Net is adaptable to optical and synthetic aperture radar (SAR) RS imagery instance segmentation tasks. It accurately predicts instance masks without relying on pixel-level labels, surpassing the segmentation accuracy of all weakly supervised methods. It also shows competitiveness when compared to hybrid and fully supervised paradigms. This research provides a low-cost, high-quality solution for the instance segmentation task in optical and SAR RS imagery.

SIRS: Multitask Joint Learning for Remote Sensing Foreground-Entity Image–Text Retrieval

SIRS: Multi-task Joint Learning for Remote Sensing Foreground-entity Image-text Retrieval

Multiscale Salient Alignment Learning for Remote-Sensing Image–Text Retrieval

Dual Modality Collaborative Learning for Cross-Source Remote Sensing Retrieval

Exploring Uni-Modal Feature Learning on Entities and Relations for Remote Sensing Cross-Modal Text-Image Retrieval

Causality-guided Step-wise Intervention and Reweighting for Remote Sensing Image Semantic Segmentation

Integrating Multisubspace Joint Learning With Multilevel Guidance for Cross-Modal Retrieval of Remote Sensing Images

Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentation

A Lightweight Multi-Scale Crossmodal Text-Image Retrieval Method in Remote Sensing

Cross-Modal Retrieval and Semantic Refinement for Remote Sensing Image Captioning

Integrating Multi-subspace Joint Learning with Multi-level Guidance for Cross-Modal Retrieval of Remote Sensing Images

A Texture and Saliency Enhanced Image Learning Method for Cross-Modal Remote Sensing Image-Text Retrieval

Knowledge-Aided Momentum Contrastive Learning for Remote-Sensing Image Text Retrieval

MCRN: A Multi-source Cross-modal Retrieval Network for Remote Sensing

Weakly Supervised Semantic Segmentation in Aerial Imagery via Cross-Image Semantic Mining

Semantic Attention and Structured Model for Weakly Supervised Instance Segmentation in Optical and SAR Remote Sensing Imagery

Spatial–Channel Attention Transformer With Pseudo Regions for Remote Sensing Image-Text Retrieval

Remote Sensing Cross-Modal Text-Image Retrieval Based on Global and Local Information

A Creative Weak Supervised Semantic Segmentation for Remote Sensing Images

Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images

Wide-Context Attention Network for Remote Sensing Image Retrieval.