Polyp-SAM: Transfer SAM for Polyp Segmentation

Yuheng Li,Mingzhe Hu,Xiaofeng Yang

2023-04-30

Abstract:Colon polyps are considered important precursors for colorectal cancer. Automatic segmentation of colon polyps can significantly reduce the misdiagnosis of colon cancer and improve physician annotation efficiency. While many methods have been proposed for polyp segmentation, training large-scale segmentation networks with limited colonoscopy data remains a challenge. Recently, the Segment Anything Model (SAM) has recently gained much attention in both natural and medical image segmentation. SAM demonstrates superior performance in several image benchmarks and therefore shows great potential for medical image segmentation. In this study, we propose Poly-SAM, a finetuned SAM model for polyp segmentation, and compare its performance to several state-of-the-art polyp segmentation models. We also compare two transfer learning strategies of SAM with and without finetuning its encoders. Evaluated on five public datasets, our Polyp-SAM achieves state-of-the-art performance on two datasets and impressive performance on three datasets, with dice scores all above 88%. This study demonstrates the great potential of adapting SAM to medical image segmentation tasks. We plan to release the code and model weights for this paper at: <a class="link-external link-https" href="https://github.com/ricklisz/Polyp-SAM" rel="external noopener nofollow">this https URL</a>.

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address the issue of automatic segmentation of colonic polyps to reduce the misdiagnosis rate of colorectal cancer and improve the efficiency of doctors' annotations. Specifically, the authors propose an improved model based on the Segment Anything Model (SAM) — Polyp-SAM, for the task of colonic polyp segmentation. Although various methods have been proposed for polyp segmentation, training large-scale segmentation networks on limited endoscopic datasets remains a challenge. SAM has garnered attention for its outstanding performance in both natural and medical image segmentation, prompting the authors to apply it to medical image segmentation and enhance its performance in colonic polyp segmentation tasks through fine-tuning. Experimental results show that Polyp-SAM achieves state-of-the-art performance on multiple public datasets, particularly demonstrating significant results on the Kvasir, CVC-ColonDB, CVC-300, and ETIS datasets. Additionally, the study explores two different transfer learning strategies and analyzes their impact on model performance. Overall, this research demonstrates the great potential of applying SAM to medical image segmentation tasks.