PE-MED: Prompt Enhancement for Interactive Medical Image Segmentation

Ao Chang,Xing Tao,Xin Yang,Yuhao Huang,Xinrui Zhou,Jiajun Zeng,Ruobing Huang,Dong Ni

2023-08-26

Abstract:Interactive medical image segmentation refers to the accurate segmentation of the target of interest through interaction (e.g., click) between the user and the image. It has been widely studied in recent years as it is less dependent on abundant annotated data and more flexible than fully automated segmentation. However, current studies have not fully explored user-provided prompt information (e.g., points), including the knowledge mined in one interaction, and the relationship between multiple interactions. Thus, in this paper, we introduce a novel framework equipped with prompt enhancement, called PE-MED, for interactive medical image segmentation. First, we introduce a Self-Loop strategy to generate warm initial segmentation results based on the first prompt. It can prevent the highly unfavorable scenarios, such as encountering a blank mask as the initial input after the first interaction. Second, we propose a novel Prompt Attention Learning Module (PALM) to mine useful prompt information in one interaction, enhancing the responsiveness of the network to user clicks. Last, we build a Time Series Information Propagation (TSIP) mechanism to extract the temporal relationships between multiple interactions and increase the model stability. Comparative experiments with other state-of-the-art (SOTA) medical image segmentation algorithms show that our method exhibits better segmentation accuracy and stability.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The paper aims to address the issue of interactive segmentation in medical image segmentation. Specifically, although current interactive segmentation methods have made significant progress, they are still insufficient in utilizing user-provided prompt information (such as click points), especially in terms of effectively mining prompt information from a single interaction and the relationships between multiple interactions. To solve these problems, the authors propose a new framework called PE-MED, which has the following three main contributions:

1. **Self-Loop Strategy**: Generates better initial segmentation results during the first user interaction, avoiding poor situations caused by insufficient initial interaction information.
2. **Prompt Attention Learning Module (PALM)**: Used to extract useful prompt information from a single interaction and enhance the network's responsiveness to user clicks.
3. **Time Series Information Propagation (TSIP)**: Used to model the continuity relationships between multiple interactions, thereby improving the model's stability.

Through these techniques, PE-MED can achieve more accurate segmentation results with fewer user interactions. Experimental results show that PE-MED outperforms existing automatic and interactive segmentation methods on two large datasets, demonstrating its superior performance in medical image segmentation tasks.