Boosting Medical Image Classification with Segmentation Foundation Model

Pengfei Gu,Zihan Zhao,Hongxiao Wang,Yaopeng Peng,Yizhe Zhang,Nishchal Sapkota,Chaoli Wang,Danny Z. Chen
2024-06-17
Abstract:The Segment Anything Model (SAM) exhibits impressive capabilities in zero-shot segmentation for natural images. Recently, SAM has gained a great deal of attention for its applications in medical image segmentation. However, to our best knowledge, no studies have shown how to harness the power of SAM for medical image classification. To fill this gap and make SAM a true ``foundation model'' for medical image analysis, it is highly desirable to customize SAM specifically for medical image classification. In this paper, we introduce SAMAug-C, an innovative augmentation method based on SAM for augmenting classification datasets by generating variants of the original images. The augmented datasets can be used to train a deep learning classification model, thereby boosting the classification performance. Furthermore, we propose a novel framework that simultaneously processes raw and SAMAug-C augmented image input, capitalizing on the complementary information that is offered by both. Experiments on three public datasets validate the effectiveness of our new approach.
Computer Vision and Pattern Recognition,Artificial Intelligence
What problem does this paper attempt to address?
The problem this paper attempts to address is: how to utilize the Segment Anything Model (SAM) to enhance the performance of medical image classification. Specifically, although SAM has shown excellent performance in natural image segmentation and has recently been used for medical image segmentation, there is currently no research demonstrating how SAM can be applied to medical image classification. Therefore, the goal of the paper is to design an effective data augmentation method based on SAM (i.e., SAMAug-C), specifically for medical image classification, by generating variants of the original images to augment the classification dataset, thereby improving the performance of the classification model. Additionally, the paper proposes a new framework that can handle both the original images and the SAMAug-C augmented images simultaneously, to fully utilize the complementary information provided by both, further enhancing the classification performance.