Abstract:Melanoma segmentation in Whole Slide Images (WSIs) is useful for prognosis and the measurement of crucial prognostic factors such as Breslow depth and primary invasive tumor size. In this paper, we present a novel approach that uses the Segment Anything Model (SAM) for automatic melanoma segmentation in microscopy slide images. Our method employs an initial semantic segmentation model to generate preliminary segmentation masks that are then used to prompt SAM. We design a dynamic prompting strategy that uses a combination of centroid and grid prompts to achieve optimal coverage of the super high-resolution slide images while maintaining the quality of generated prompts. To optimize for invasive melanoma segmentation, we further refine the prompt generation process by implementing in-situ melanoma detection and low-confidence region filtering. We select Segformer as the initial segmentation model and EfficientSAM as the segment anything model for parameter-efficient fine-tuning. Our experimental results demonstrate that this approach not only surpasses other state-of-the-art melanoma segmentation methods but also significantly outperforms the baseline Segformer by 9.1% in terms of IoU.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper aims to address the problem of melanoma segmentation in microscope slide images. Specifically, the authors propose a new method that utilizes the Segment Anything Model (SAM) to automatically segment melanomas. This task is of significant importance for the prognosis of melanoma and the measurement of key prognostic factors such as Breslow depth and the size of the primary invasive tumor. ### Background and Motivation Melanoma is a severe form of skin cancer that can be classified into in situ melanoma and invasive melanoma based on its progression and location. In situ melanoma is confined to the epidermis, while invasive melanoma penetrates the epidermis into the dermis, posing a risk of spreading to other vital organs. Early detection and accurate diagnosis of melanoma are crucial for improving survival rates. Traditional diagnostic methods require examining tissue specimens under a microscope, but the advent of whole slide images (WSI) has digitized this process, providing high-resolution digital slide images. These images help pathologists examine cellular structures, Breslow depth, and complex histological features to determine the staging and invasiveness of melanoma. In recent years, many studies have attempted to use deep learning techniques for melanoma segmentation, but existing methods still have limitations when dealing with high-resolution microscope slide images. Therefore, this paper proposes a new method that combines the initial segmentation masks generated by a semantic segmentation model to prompt SAM for more accurate melanoma segmentation. ### Method Overview 1. **Initial Mask Generation**: Use Segformer to generate initial segmentation masks and optimize the masks through in situ melanoma detection and low-confidence region filtering. 2. **Prompt Generation**: Generate single-point prompts from the optimized masks using a dynamic prompting strategy, including centroid prompts and grid prompts. 3. **Final Mask Generation**: Run SAM to generate the final invasive melanoma masks and combine them with the initial masks to improve segmentation accuracy and robustness. ### Experimental Results The experimental results show that this method not only surpasses other state-of-the-art melanoma segmentation methods but also improves the IoU by 9.1% compared to the baseline Segformer. Additionally, through a series of ablation experiments, the authors validated the contribution of each component to the overall performance, particularly in in situ melanoma detection and low-confidence region filtering. ### Conclusion The method proposed in this paper achieves significant performance improvements in the task of melanoma segmentation in microscope slide images, providing strong support for automatic diagnosis and assisting in the manual measurement of key prognostic factors.

Adapting Segment Anything Model to Melanoma Segmentation in Microscopy Slide Images

Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging

Deep Learning-Based Semantic Segmentation of Non-Melanocytic Skin Tumors in Whole-Slide Histopathological Images.

Segment Anything Model for Medical Image Analysis: an Experimental Study

WSI-SAM: Multi-resolution Segment Anything Model (SAM) for histopathology whole-slide images

Unsupervised sub-segmentation for pigmented skin lesions.

SAM-Path: A Segment Anything Model for Semantic Segmentation in Digital Pathology

SkinSAM: Empowering Skin Cancer Segmentation with Segment Anything Model

How Segment Anything Model (SAM) Boost Medical Image Segmentation?

Evaluation and improvement of Segment Anything Model for interactive histopathology image segmentation

SAM Fails to Segment Anything? – SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and More

Mask-Enhanced Segment Anything Model for Tumor Lesion Semantic Segmentation

Segment Anything in Medical Images

Towards Segment Anything Model (SAM) for Medical Image Segmentation: A Survey

Segment anything model 2: an application to 2D and 3D medical images

Segment anything model for medical image segmentation: Current applications and future directions

Part-aware Personalized Segment Anything Model for Patient-Specific Segmentation

Segment Anything Model for Medical Images?

S-SAM: SVD-based Fine-Tuning of Segment Anything Model for Medical Image Segmentation