Abstract:Instance segmentation in remote sensing (RS) imagery aims to predict the locations of instances and represent them with pixel-level masks. Thanks to the more accurate pixel-level information for each instance, instance segmentation has enormous potential applications in resource planning, urban surveillance, and military reconnaissance. However, current RS imagery instance segmentation methods mostly follow the fully supervised paradigm, relying on expensive pixel-level labels. Moreover, remote sensing imagery suffers from cluttered backgrounds and significant variations in target scales, making segmentation challenging. To accommodate these limitations, we propose a semantic attention enhancement and structured model-guided multi-scale weakly supervised instance segmentation network (SASM-Net). Building upon the modeling of spatial relationships for weakly supervised instance segmentation, we further design the multi-scale feature extraction module (MSFE module), semantic attention enhancement module (SAE module), and structured model guidance module (SMG module) for SASM-Net to enable a balance between label production costs and visual processing. The MSFE module adopts a hierarchical approach similar to the residual structure to establish equivalent feature scales and to adapt to the significant scale variations of instances in RS imagery. The SAE module is a dual-stream structure with semantic information prediction and attention enhancement streams. It can enhance the network's activation of instances in the images and reduce cluttered backgrounds' interference. The SMG module can assist the SAE module in the training process to construct supervision with edge information, which can implicitly lead the model to a representation with structured inductive bias, reducing the impact of the low sensitivity of the model to edge information caused by the lack of fine-grained pixel-level labeling. Experimental results indicate that the proposed SASM-Net is adaptable to optical and synthetic aperture radar (SAR) RS imagery instance segmentation tasks. It accurately predicts instance masks without relying on pixel-level labels, surpassing the segmentation accuracy of all weakly supervised methods. It also shows competitiveness when compared to hybrid and fully supervised paradigms. This research provides a low-cost, high-quality solution for the instance segmentation task in optical and SAR RS imagery.

Weakly Supervised Segmentation of SAR Imagery Using Superpixel and Hierarchically Adversarial CRF

SAR Image segmentation based on convolutional-wavelet neural network and markov random field

A fast, weighted CRF algorithm based on a two-step superpixel generation for SAR image segmentation

Pixel-Wise Segmentation Of Sar Imagery Using Encoder-Decoder Network And Fully-Connected Crf

Synthetic Aperture Radar Image Segmentation Based On Improved Fuzzy Markov Random Field Model

SAR Image Segmentation Based on Hierarchical Visual Semantic and Adaptive Neighborhood Multinomial Latent Model

Adaptive Hierarchical Multinomial Latent Model with Hybrid Kernel Function for SAR Image Semantic Segmentation

SAR Image Segmentation Based on Fisher Vector Superpixel Generation and Label Revision

SAR Image Segmentation Based on Constrained Smoothing and Hierarchical Label Correction

SAR Image Classification Using Fully Connected Conditional Random Fields Combined with Deep Learning and Superpixel Boundary Constraint

A Semantic Segmentation Method for SAR Image with Assistance of Self-Supervised Scene Classification

Cross-Modality Features Fusion for Synthetic Aperture Radar Image Segmentation

Semantic Attention and Structured Model for Weakly Supervised Instance Segmentation in Optical and SAR Remote Sensing Imagery

CCNR: Cross-regional Context and Noise Regularization for SAR Image Segmentation

Semantic Segmentation For Sar Image Based On Texture Complexity Analysis And Key Superpixels

Weakly Supervised Semantic Segmentation by Iterative Superpixel-CRF Refinement with Initial Clues Guiding

Superpixel Consistency Saliency Map Generation for Weakly Supervised Semantic Segmentation of Remote Sensing Images

SAR Image Segmentation Based on Complicated Region-Sensitive Adaptive Superpixel Generation and Hybrid Edge Correction

Fast Algorithm Based on Superpixel‐level Conditional Triplet Markov Field for Successive‐approximation Resistor Image Segmentation

Unreliable Pixel Contrast Based on von Mises–Fisher Distribution for Semi-Supervised SAR Segmentation

Structural Feature Learning-Based Unsupervised Semantic Segmentation of Synthetic Aperture Radar Image