Point-supervised Brain Tumor Segmentation with Box-prompted MedSAM

Xiaofeng Liu,Jonghye Woo,Chao Ma,Jinsong Ouyang,Georges El Fakhri
2024-08-02
Abstract:Delineating lesions and anatomical structure is important for image-guided interventions. Point-supervised medical image segmentation (PSS) has great potential to alleviate costly expert delineation labeling. However, due to the lack of precise size and boundary guidance, the effectiveness of PSS often falls short of expectations. Although recent vision foundational models, such as the medical segment anything model (MedSAM), have made significant advancements in bounding-box-prompted segmentation, it is not straightforward to utilize point annotation, and is prone to semantic ambiguity. In this preliminary study, we introduce an iterative framework to facilitate semantic-aware point-supervised MedSAM. Specifically, the semantic box-prompt generator (SBPG) module has the capacity to convert the point input into potential pseudo bounding box suggestions, which are explicitly refined by the prototype-based semantic similarity. This is then succeeded by a prompt-guided spatial refinement (PGSR) module that harnesses the exceptional generalizability of MedSAM to infer the segmentation mask, which also updates the box proposal seed in SBPG. Performance can be progressively improved with adequate iterations. We conducted an evaluation on BraTS2018 for the segmentation of whole brain tumors and demonstrated its superior performance compared to traditional PSS methods and on par with box-supervised methods.
Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning,Image and Video Processing,Medical Physics
What problem does this paper attempt to address?
This paper aims to solve the point - supervised segmentation (PSS) problem in medical image segmentation, especially the segmentation of brain tumors. Although point - supervised methods have the potential to reduce the cost of expert annotation, their performance is usually not as good as methods using mask or bounding - box supervision, because point - supervision lacks precise guidance on the target size and boundary. In addition, existing vision - based models such as MedSAM have made significant progress in bounding - box - prompted segmentation tasks, but directly using point annotations has the problem of semantic ambiguity, and these models lack classification ability, resulting in the inability to accurately segment specific lesions or structures. To overcome these problems, the authors propose an iterative framework to improve the MedSAM model under point - supervision by introducing the **Semantic Box - Prompt Generator (SBPG)** module and the **Prompt - Guided Spatial Refinement (PGSR)** module. Specifically, the SBPG module can convert point inputs into potential pseudo - bounding - box proposals and explicitly optimize these bounding - boxes based on prototype - based semantic similarity. Subsequently, the PGSR module uses the strong generalization ability of MedSAM to predict segmentation masks and update the bounding - box proposal seeds in SBPG. Through multiple iterations, the performance can be gradually improved. The experimental results show that on the BraTS2018 dataset, this method performs excellently in the whole - brain - tumor - segmentation task, outperforming traditional point - supervised methods and approaching the performance of bounding - box - supervised methods. This indicates that through appropriate iteration, point - prompts can achieve an effect comparable to that of bounding - box - prompted MedSAM.