Abstract:Segment Anything Model (SAM) has demonstrated impressive performance on a wide range of natural image segmentation tasks. However, its performance significantly deteriorates when directly applied to medical domain, due to the remarkable differences between natural images and medical images. Some researchers have attempted to train SAM on large scale medical datasets. However, poor zero-shot performance is observed from the experimental results. In this context, inspired by the superior performance of U-Net-like models in medical image segmentation, we propose SAMUNet, a new foundation model which incorporates U-Net to the original SAM, to fully leverage the powerful contextual modeling ability of convolutions. To be specific, we parallel a convolutional branch in the image encoder, which is trained independently with the vision Transformer branch frozen. Additionally, we employ multi-scale fusion in the mask decoder, to facilitate accurate segmentation of objects with different scales. We train SAM-UNet on SA-Med2D-16M, the largest 2-dimensional medical image segmentation dataset to date, yielding a universal pretrained model for medical images. Extensive experiments are conducted to evaluate the performance of the model, and state-of-the-art result is achieved, with a dice similarity coefficient score of 0.883 on SA-Med2D-16M dataset. Specifically, in zero-shot segmentation experiments, our model not only significantly outperforms previous large medical SAM models across all modalities, but also substantially mitigates the performance degradation seen on unseen modalities. It should be highlighted that SAM-UNet is an efficient and extensible foundation model, which can be further fine-tuned for other downstream tasks in medical community. The code is available at <a class="link-external link-https" href="https://github.com/Hhankyangg/sam-unet" rel="external noopener nofollow">this https URL</a>.

Universal Organizer of Segment Anything Model for Unsupervised Semantic Segmentation

Universal Organizer of SAM for Unsupervised Semantic Segmentation

Adapting Segment Anything Model for Unseen Object Instance Segmentation

Segment Anything without Supervision

Unsupervised Universal Image Segmentation

UVOSAM: A Mask-free Paradigm for Unsupervised Video Object Segmentation via Segment Anything Model

SAM-Assisted Remote Sensing Imagery Semantic Segmentation With Object and Boundary Constraints

SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentation

Semantic-SAM: Segment and Recognize Anything at Any Granularity

AM-SAM: Automated Prompting and Mask Calibration for Segment Anything Model

SEG-SAM: Semantic-Guided SAM for Unified Medical Image Segmentation

Segment Any Object Model (SAOM): Real-to-Simulation Fine-Tuning Strategy for Multi-Class Multi-Instance Segmentation

RSAM-Seg: A SAM-based Approach with Prior Knowledge Integration for Remote Sensing Image Semantic Segmentation

MeSAM: Multiscale Enhanced Segment Anything Model for Optical Remote Sensing Images

Effective SAM Combination for Open-Vocabulary Semantic Segmentation

Segment Anything in High Quality

Multi-Scale and Detail-Enhanced Segment Anything Model for Salient Object Detection

SAM-UNet:Enhancing Zero-Shot Segmentation of SAM for Universal Medical Images

Segment Anything with Multiple Modalities

Enhancing Unsupervised Semantic Segmentation Through Context-Aware Clustering