SQA-SAM: Segmentation Quality Assessment for Medical Images Utilizing the Segment Anything Model

Yizhe Zhang,Shuo Wang,Tao Zhou,Qi Dou,Danny Z. Chen

2023-12-15

Abstract:Segmentation quality assessment (SQA) plays a critical role in the deployment of a medical image based AI system. Users need to be informed/alerted whenever an AI system generates unreliable/incorrect predictions. With the introduction of the Segment Anything Model (SAM), a general foundation segmentation model, new research opportunities emerged in how one can utilize SAM for medical image segmentation. In this paper, we propose a novel SQA method, called SQA-SAM, which exploits SAM to enhance the accuracy of quality assessment for medical image segmentation. When a medical image segmentation model (MedSeg) produces predictions for a test image, we generate visual prompts based on the predictions, and SAM is utilized to generate segmentation maps corresponding to the visual prompts. How well MedSeg's segmentation aligns with SAM's segmentation indicates how well MedSeg's segmentation aligns with the general perception of objectness and image region partition. We develop a score measure for such alignment. In experiments, we find that the generated scores exhibit moderate to strong positive correlation (in Pearson correlation and Spearman correlation) with Dice coefficient scores reflecting the true segmentation quality.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to evaluate the segmentation quality in medical image segmentation. Specifically, when a medical image segmentation model (MedSeg) makes predictions on test images, users need to be informed or reminded whether the AI system has generated unreliable or incorrect predictions. Existing medical AI systems often produce unreliable results when encountering samples outside the training data distribution, and usually lack the evaluation of segmentation quality, which leads to doctors' lack of confidence when actually using these AI systems.

To solve this problem, the author proposes a new segmentation quality assessment (SQA) method, named SQA - SAM. This method utilizes the Segment Anything Model (SAM), which is a general basic segmentation model, to enhance the accuracy of quality assessment in medical image segmentation by generating segmentation maps corresponding to visual cues. SQA - SAM measures the consistency between the segmentation results of MedSeg and general object perception and image region division by comparing the matching degree between the segmentation results of MedSeg and the segmentation maps generated by SAM, and develops a scoring metric based on this consistency.

The experimental results show that there is a moderate to strong positive correlation (Pearson correlation and Spearman correlation) between the scores generated by SQA - SAM and the Dice coefficient scores that reflect the real segmentation quality, indicating the effectiveness of this method in evaluating the quality of medical image segmentation.