Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directions

Yichi Zhang,Zhenrong Shen,Rushi Jiao

2024-01-07

Abstract:Due to the inherent flexibility of prompting, foundation models have emerged as the predominant force in the fields of natural language processing and computer vision. The recent introduction of the Segment Anything Model (SAM) signifies a noteworthy expansion of the prompt-driven paradigm into the domain of image segmentation, thereby introducing a plethora of previously unexplored capabilities. However, the viability of its application to medical image segmentation remains uncertain, given the substantial distinctions between natural and medical images. In this work, we provide a comprehensive overview of recent endeavors aimed at extending the efficacy of SAM to medical image segmentation tasks, encompassing both empirical benchmarking and methodological adaptations. Additionally, we explore potential avenues for future research directions in SAM's role within medical image segmentation. While direct application of SAM to medical image segmentation does not yield satisfactory performance on multi-modal and multi-target medical datasets so far, numerous insights gleaned from these efforts serve as valuable guidance for shaping the trajectory of foundational models in the realm of medical image analysis. To support ongoing research endeavors, we maintain an active repository that contains an up-to-date paper list and a succinct summary of open-source projects at

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper primarily explores the application and challenges of the Segment Anything Model (SAM) in the field of medical image segmentation and proposes a series of improvement methods. Specifically:

1. **Evaluating SAM's Zero-Shot Performance in Medical Image Segmentation**:
- Researchers evaluated SAM's zero-shot segmentation performance across multiple medical imaging modalities (such as CT, MRI, pathology images, etc.).
- Experimental results indicate that although SAM performs well in some cases, it struggles with targets that have complex structures, low contrast, or irregular shapes.

2. **Adaptive Improvements to SAM for Medical Image Segmentation Tasks**:
- To address the poor performance of SAM when directly applied to medical image segmentation, researchers proposed various improvement methods, including full fine-tuning, parameter-efficient fine-tuning, and automatic prompt generation.
- Some of these methods (such as MedSAM, Med-SA, etc.) improve SAM's performance in medical image segmentation by fine-tuning specific modules or introducing adaptive layers.
- Other methods (such as MedLSAM) focus on automatically generating high-quality prompts to reduce reliance on manual annotations.

Through these efforts, researchers aim to enhance the generality and accuracy of SAM in medical image segmentation tasks, enabling it to better handle the challenges posed by different modalities and complex datasets.