Abstract:Background: Large foundation models, such as the Segment Anything Model (SAM), have shown remarkable performance in image segmentation tasks. However, the optimal approach to achieve true utility of these models for domain-specific applications, such as medical image segmentation, remains an open question. Recent studies have released a medical version of the foundation model MedSAM by training on vast medical data, who promised SOTA medical segmentation. Independent community inspection and dissection is needed. Purpose: This study assesses the performance of off-the-shelf medical foundation model MedSAM for the segmentation of anatomical structures in pelvic MR images. We also evaluate the dependency on prompting scheme and demonstrate the gain of further specialized fine-tuning. Methods: MedSAM and its lightweight version LiteMedSAM were evaluated out-of-the-box on a public MR dataset consisting of 589 pelvic images split 80:20 for training and testing. An nnU-Net model was trained from scratch to serve as a benchmark and to provide bounding box prompts for MedSAM. MedSAM was evaluated using different quality bounding boxes, those derived from ground truth labels, those derived from nnU-Net, and those derived from the former two but with 5-pixel isometric expansion. Lastly, LiteMedSAM was refined on the training set and reevaluated on this task. Results: Out-of-the-box MedSAM and LiteMedSAM both performed poorly across the structure set, especially for disjoint or non-convex structures. Varying prompt with different bounding box inputs had minimal effect. The mean Dice score and mean Hausdorff distances (in mm) for obturator internus using MedSAM and LiteMedSAM were {0.251 +/- 0.110, 0.101 +/- 0.079} and {34.142 +/- 5.196, 33.688 +/- 5.306}, respectively. Fine-tuning of LiteMedSAM led to significant performance gain, improving Dice score and Hausdorff distance for the obturator internus to 0.864 +/- 0.123 and 5.022 +/- 10.684, on par with nnU-Net with no significant difference in evaluation of most structures. All segmentation structures all benefited significantly from specialized refinement, at varying improvement margin. Conclusion: Our study alludes to the potential of deep learning models like MedSAM and lite MedSAM for medical segmentation but also highlight the need for specialized refinement and adjudication: it is quite likely that off-the-shelf use of such large foundation models may be suboptimal, and specialized fine-tuning can significantly enhance segmentation accuracy.

Effect of diet and tylosin on chronic diarrhea in beagles.

SAM-Path: A Segment Anything Model for Semantic Segmentation in Digital Pathology

G-SAM: GMM-based segment anything model for medical image classification and segmentation

AGSAM: Agent-Guided Segment Anything Model for Automatic Segmentation in Few-Shot Scenarios

UN-SAM: Universal Prompt-Free Segmentation for Generalized Nuclei Images

Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging

Segment Anything Model for Medical Image Analysis: an Experimental Study

DeSAM: Decoupled Segment Anything Model for Generalizable Medical Image Segmentation

Necessity and Impact of Specialization of Large Foundation Model for Medical Segmentation Tasks

Multi-Prompt Fine-Tuning of Foundation Models for Enhanced Medical Image Segmentation

Customized Segment Anything Model for Medical Image Segmentation

nnSAM: Plug-and-play Segment Anything Model Improves nnUNet Performance

CellSAM: Advancing Pathologic Image Cell Segmentation via Asymmetric Large‐Scale Vision Model Feature Distillation Aggregation Network

Point-supervised Brain Tumor Segmentation with Box-prompted MedSAM

Feature-prompting GBMSeg: One-Shot Reference Guided Training-Free Prompt Engineering for Glomerular Basement Membrane Segmentation

MA-SAM: Modality-agnostic SAM adaptation for 3D medical image segmentation

Beyond Adapting SAM: Towards End-to-End Ultrasound Image Segmentation via Auto Prompting

S-SAM: SVD-based Fine-Tuning of Segment Anything Model for Medical Image Segmentation

Repurposing Traditional U-Net Predictions for Sparse SAM Prompting in Medical Image Segmentation

Towards a Comprehensive, Efficient and Promptable Anatomic Structure Segmentation Model using 3D Whole-body CT Scans

SAM-MPA: Applying SAM to Few-shot Medical Image Segmentation using Mask Propagation and Auto-prompting