Cross-Organ Domain Adaptive Neural Network for Pancreatic Endoscopic Ultrasound Image Segmentation

ZhiChao Yan,Hui Xue,Yi Zhu,Bin Xiao,Hao Yuan

DOI: https://doi.org/10.48550/arXiv.2409.04718

2024-09-07

Abstract:Accurate segmentation of lesions in pancreatic endoscopic ultrasound (EUS) images is crucial for effective diagnosis and treatment. However, the collection of enough crisp EUS images for effective diagnosis is arduous. Recently, domain adaptation (DA) has been employed to address these challenges by leveraging related knowledge from other domains. Most DA methods only focus on multi-view representations of the same organ, which makes it still tough to clearly depict the tumor lesion area with limited semantic information. Although transferring homogeneous similarity from different organs could benefit the issue, there is a lack of relevant work due to the enormous domain gap between them. To address these challenges, we propose the Cross-Organ Tumor Segmentation Networks (COTS-Nets), consisting of a universal network and an auxiliary network. The universal network utilizes boundary loss to learn common boundary information of different tumors, enabling accurate delineation of tumors in EUS despite limited and low-quality data. Simultaneously, we incorporate consistency loss in the universal network to align the prediction of pancreatic EUS with tumor boundaries from other organs to mitigate the domain gap. To further reduce the cross-organ domain gap, the auxiliary network integrates multi-scale features from different organs, aiding the universal network in acquiring domain-invariant knowledge. Systematic experiments demonstrate that COTS-Nets significantly improves the accuracy of pancreatic cancer diagnosis. Additionally, we developed the Pancreatic Cancer Endoscopic Ultrasound (PCEUS) dataset, comprising 501 pathologically confirmed pancreatic EUS images, to facilitate model development.

Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

### Problems the paper attempts to solve

This paper aims to solve the problem of accurately segmenting the lesion area in pancreatic endoscopic ultrasound (EUS) images. Specifically, the paper focuses on the following aspects:

1. **Data scarcity**: Obtaining high - quality pancreatic EUS images is very difficult because it requires precise operations by professional doctors. The scarcity of data limits the training of the model and the improvement of performance.
2. **Low - quality images**: The boundaries between tumors and surrounding normal tissues in EUS images are often not clear, making the segmentation task particularly challenging.
3. **Cross - organ domain adaptation**: Existing domain adaptation methods mainly focus on multi - view or multi - modal representations of the same organ and have limited effects on knowledge transfer between different organs. The large domain gap between different organs makes it difficult to apply these methods directly and effectively.

To address the above challenges, the paper proposes the Cross - Organ Tumor Segmentation Networks (COTS - Nets), which consists of a general network and an auxiliary network. The general network learns the common boundary information of different tumors by introducing the boundary loss, thereby achieving accurate tumor segmentation with limited data. At the same time, the prediction of pancreatic EUS is aligned with the tumor boundaries from other organs through the consistency loss to reduce the domain gap. The auxiliary network helps the general network obtain domain - invariant knowledge by integrating multi - scale features, further reducing the cross - organ domain gap.

### Main contributions

1. **Cross - organ paradigm**: A cross - organ paradigm that mimics the professional training process of radiologists is proposed, enabling the model to learn inherent diagnostic features such as echoes, tumor boundaries, and textures from images of different organs.
2. **Novel COTS - Nets framework**: The general network utilizes tumor - boundary - related information to accurately depict EUS lesion areas with limited data. The auxiliary network enhances representation learning by fusing multi - scale features and refines domain - invariant knowledge.
3. **New dataset**: A new dataset for endoscopic ultrasound image segmentation of pancreatic cancer, "PCEUS", is constructed, which contains 501 pathologically confirmed EUS images for model development and validation.

Through these innovations, the paper significantly improves the accuracy of pancreatic cancer diagnosis and provides a new solution for cross - organ medical image segmentation.