Abstract:Segment Anything Model (SAM) is an advanced foundational model for image segmentation, widely applied to remote sensing images (RSIs). Due to the domain gap between RSIs and natural images, traditional methods typically use SAM as a source pre-trained model and fine-tune it with fully supervised masks. Unlike these methods, our work focuses on fine-tuning SAM using more convenient and challenging point annotations. Leveraging SAM's zero-shot capabilities, we adopt a self-training framework that iteratively generates pseudo-labels for training. However, if the pseudo-labels contain noisy labels, there is a risk of error accumulation. To address this issue, we extract target prototypes from the target dataset and use the Hungarian algorithm to match them with prediction prototypes, preventing the model from learning in the wrong direction. Additionally, due to the complex backgrounds and dense distribution of objects in RSI, using point prompts may result in multiple objects being recognized as one. To solve this problem, we propose a negative prompt calibration method based on the non-overlapping nature of instance masks. In brief, we use the prompts of overlapping masks as corresponding negative signals, resulting in refined masks. Combining the above methods, we propose a novel Pointly-supervised Segment Anything Model named PointSAM. We conduct experiments on RSI datasets, including WHU, HRSID, and NWPU VHR-10, and the results show that our method significantly outperforms direct testing with SAM, SAM2, and other comparison methods. Furthermore, we introduce PointSAM as a point-to-box converter and achieve encouraging results, suggesting that this method can be extended to other point-supervised tasks. The code is available at <a class="link-external link-https" href="https://github.com/Lans1ng/PointSAM" rel="external noopener nofollow">this https URL</a>.

SAM-RSIS: Progressively Adapting SAM With Box Prompting to Remote Sensing Image Instance Segmentation

RSAM-Seg: A SAM-based Approach with Prior Knowledge Integration for Remote Sensing Image Semantic Segmentation

MeSAM: Multiscale Enhanced Segment Anything Model for Optical Remote Sensing Images

Tuning a SAM-Based Model with Multi-Cognitive Visual Adapter to Remote Sensing Instance Segmentation

PointSAM: Pointly-Supervised Segment Anything Model for Remote Sensing Images

RSPS-SAM: A Remote Sensing Image Panoptic Segmentation Method Based on SAM

RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation Based on Visual Foundation Model

SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model

AM-SAM: Automated Prompting and Mask Calibration for Segment Anything Model

The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot

SAM Fails to Segment Anything? – SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and More

Multi-view Remote Sensing Image Segmentation With SAM priors

SAM-Adapter: Adapting Segment Anything in Underperformed Scenes

SAM-Assisted Remote Sensing Imagery Semantic Segmentation with Object and Boundary Constraints

PA-SAM: Prompt Adapter SAM for High-Quality Image Segmentation

Self-guided Few-shot Semantic Segmentation for Remote Sensing Imagery Based on Large Vision Models

MapSAM: Adapting Segment Anything Model for Automated Feature Detection in Historical Maps

Evaluation Study on SAM 2 for Class-agnostic Instance-level Segmentation

Prompting DirectSAM for Semantic Contour Extraction in Remote Sensing Images

MaskSAM: Towards Auto-prompt SAM with Mask Classification for Medical Image Segmentation