Is Segment Anything Model Ready for X-Ray Object Segmentation?

Jingxue Hu,Yuehu Liu,Chi Zhang,Zhichao Cui

DOI: https://doi.org/10.1109/cac59555.2023.10451407

2023-01-01

Abstract:Since public transportation scenarios have high requirements for baggage security, accurate X-Ray Object Segmentation for baggage security inspection is vital to help maintain social security. Since X-Ray images generated by different X-ray security machines differ in terms of color schemes, pixels and geometric deformation, detection and segmentation methods trained on certain datasets generalize poorly in out-of-distribution X-ray images. Recently, the Segment Anything Model (SAM) has been published as a foundation model for image segmentation, which achieves great generalization ability on natural images based on its 1B mask training dataset. In this paper, we propose to explore whether such huge foundation model exist zero-shot segmentation capabilities on X-ray images like humans do. Compared with the simple experimental protocol (only positive point prompt mode involved) in the original SAM paper, we test its performance sufficiently on two X-ray prohibited datasets with a total of 37524 images, including 58304 instances in 16 categories, when categories, occlusion conditions, prompt modes and checkpoints vary. Moreover, we compared SAM's performance with other SOTA detection models and segmentation models. Our empirical study shows that though SAM has certain segmentation ability on X-ray images especially with prompts, the performance on irregular instances and complex overlapping conditions still wait to be improved due to the overlapping characteristics.