Abstract:Segment Anything Model (SAM) has demonstrated impressive performance on a wide range of natural image segmentation tasks. However, its performance significantly deteriorates when directly applied to medical domain, due to the remarkable differences between natural images and medical images. Some researchers have attempted to train SAM on large scale medical datasets. However, poor zero-shot performance is observed from the experimental results. In this context, inspired by the superior performance of U-Net-like models in medical image segmentation, we propose SAMUNet, a new foundation model which incorporates U-Net to the original SAM, to fully leverage the powerful contextual modeling ability of convolutions. To be specific, we parallel a convolutional branch in the image encoder, which is trained independently with the vision Transformer branch frozen. Additionally, we employ multi-scale fusion in the mask decoder, to facilitate accurate segmentation of objects with different scales. We train SAM-UNet on SA-Med2D-16M, the largest 2-dimensional medical image segmentation dataset to date, yielding a universal pretrained model for medical images. Extensive experiments are conducted to evaluate the performance of the model, and state-of-the-art result is achieved, with a dice similarity coefficient score of 0.883 on SA-Med2D-16M dataset. Specifically, in zero-shot segmentation experiments, our model not only significantly outperforms previous large medical SAM models across all modalities, but also substantially mitigates the performance degradation seen on unseen modalities. It should be highlighted that SAM-UNet is an efficient and extensible foundation model, which can be further fine-tuned for other downstream tasks in medical community. The code is available at <a class="link-external link-https" href="https://github.com/Hhankyangg/sam-unet" rel="external noopener nofollow">this https URL</a>.

ProtoSAM: One-Shot Medical Image Segmentation With Foundational Models

SAM.MD: Zero-shot medical image segmentation capabilities of the Segment Anything Model

SimSAM: Zero-shot Medical Image Segmentation via Simulated Interaction

SAM3D: Zero-Shot Semi-Automatic Segmentation in 3D Medical Images with the Segment Anything Model

SAM-UNet:Enhancing Zero-Shot Segmentation of SAM for Universal Medical Images

Dr-SAM: U-Shape Structure Segment Anything Model for Generalizable Medical Image Segmentation

No More Training: SAM's Zero-Shot Transfer Capabilities for Cost-Efficient Medical Image Segmentation

SAM-MPA: Applying SAM to Few-shot Medical Image Segmentation using Mask Propagation and Auto-prompting

Pathological Primitive Segmentation Based on Visual Foundation Model with Zero-Shot Mask Generation

Medical SAM 2: Segment medical images as video via Segment Anything Model 2

Interactive 3D Medical Image Segmentation with SAM 2

SAM Fewshot Finetuning for Anatomical Segmentation in Medical Images

Customized Segment Anything Model for Medical Image Segmentation

AGSAM: Agent-Guided Segment Anything Model for Automatic Segmentation in Few-Shot Scenarios

Retrieval-augmented Few-shot Medical Image Segmentation with Foundation Models

Input Augmentation with SAM: Boosting Medical Image Segmentation with Segmentation Foundation Model

Medical Image Segmentation with SAM-generated Annotations

Zero-shot performance of the Segment Anything Model (SAM) in 2D medical imaging: A comprehensive evaluation and practical guidelines

Self-Sampling Meta SAM: Enhancing Few-shot Medical Image Segmentation with Meta-Learning

Segment Anything Model (SAM) for Medical Image Segmentation: A Preliminary Review

Segment anything model 2: an application to 2D and 3D medical images