Abstract:Segment Anything Model (SAM) is an advanced foundational model for image segmentation, widely applied to remote sensing images (RSIs). Due to the domain gap between RSIs and natural images, traditional methods typically use SAM as a source pre-trained model and fine-tune it with fully supervised masks. Unlike these methods, our work focuses on fine-tuning SAM using more convenient and challenging point annotations. Leveraging SAM's zero-shot capabilities, we adopt a self-training framework that iteratively generates pseudo-labels for training. However, if the pseudo-labels contain noisy labels, there is a risk of error accumulation. To address this issue, we extract target prototypes from the target dataset and use the Hungarian algorithm to match them with prediction prototypes, preventing the model from learning in the wrong direction. Additionally, due to the complex backgrounds and dense distribution of objects in RSI, using point prompts may result in multiple objects being recognized as one. To solve this problem, we propose a negative prompt calibration method based on the non-overlapping nature of instance masks. In brief, we use the prompts of overlapping masks as corresponding negative signals, resulting in refined masks. Combining the above methods, we propose a novel Pointly-supervised Segment Anything Model named PointSAM. We conduct experiments on RSI datasets, including WHU, HRSID, and NWPU VHR-10, and the results show that our method significantly outperforms direct testing with SAM, SAM2, and other comparison methods. Furthermore, we introduce PointSAM as a point-to-box converter and achieve encouraging results, suggesting that this method can be extended to other point-supervised tasks. The code is available at <a class="link-external link-https" href="https://github.com/Lans1ng/PointSAM" rel="external noopener nofollow">this https URL</a>.

Correcting Pseudo Labels in Semi Supervised Object Detection with SAM

Pseudo-label Correction and Learning For Semi-Supervised Object Detection

Rethinking Pseudo Labels for Semi-Supervised Object Detection

Twin Pseudo-training for Semi-Supervised Semantic Segmentation

Push the Boundary of SAM: A Pseudo-label Correction Framework for Medical Segmentation

PseCo: Pseudo Labeling and Consistency Training for Semi-Supervised Object Detection

Weakly supervised salient object detection via bounding-box annotation and SAM model

Don’t Turn a Blind Eye to Localization Noise: Localization Pseudo-label Correction and Learning for Semi-Supervised Object Detection

Mind the Gap: Polishing Pseudo labels for Accurate Semi-supervised Object Detection

Robust Teacher: Self-correcting Pseudo-Label-guided Semi-Supervised Learning for Object Detection

Learning Pseudo Labels for Semi-and-weakly Supervised Semantic Segmentation

Mixed Pseudo Labels for Semi-Supervised Object Detection

PointSAM: Pointly-Supervised Segment Anything Model for Remote Sensing Images

WeakSAM: Segment Anything Meets Weakly-supervised Instance-level Recognition

SAM-Induced Pseudo Fully Supervised Learning for Weakly Supervised Object Detection in Remote Sensing Images

USD: U ncertainty-based One-phase Learning to Enhance Pseudo-Label Reliability for S emi-Supervised Object D etection

SAM Carries the Burden: A Semi-Supervised Approach Refining Pseudo Labels for Medical Segmentation

SemiSAM: Enhancing Semi-Supervised Medical Image Segmentation via SAM-Assisted Consistency Regularization

Unsupervised Domain Adaptive Salient Object Detection Through Uncertainty-Aware Pseudo-Label Learning

Semi-Supervised Text Detection with Accurate Pseudo-Labels

Refining and reweighting pseudo labels for weakly supervised object detection