Abstract:Automatic gastric tumor segmentation and lymph node (LN) classification not only can assist radiologists in reading images, but also provide image-guided clinical diagnosis and improve diagnosis accuracy. However, due to the inhomogeneous intensity distribution of gastric tumor and LN in CT scans, the ambiguous/missing boundaries, and highly variable shapes of gastric tumor, it is quite challenging to develop an automatic solution. To comprehensively address these challenges, we propose a novel 3D multi-attention guided multi-task learning network for simultaneous gastric tumor segmentation and LN classification, which makes full use of the complementary information extracted from different dimensions, scales, and tasks. Specifically, we tackle task correlation and heterogeneity with the convolutional neural network consisting of scale-aware attention-guided shared feature learning for refined and universal multi-scale features, and task-aware attention-guided feature learning for task-specific discriminative features. This shared feature learning is equipped with two types of scale-aware attention (visual attention and adaptive spatial attention) and two stage-wise deep supervision paths. The task-aware attention-guided feature learning comprises a segmentation-aware attention module and a classification-aware attention module. The proposed 3D multi-task learning network can balance all tasks by combining segmentation and classification loss functions with weight uncertainty. We evaluate our model on an in-house CT images dataset collected from three medical centers. Experimental results demonstrate that our method outperforms the state-of-the-art algorithms, and obtains promising performance for tumor segmentation and LN classification. Moreover, to explore the generalization for other segmentation tasks, we also extend the proposed network to liver tumor segmentation in CT images of the MICCAI 2017 Liver Tumor Segmentation Challenge. Our implementation is released at https://github.com/infinite-tao/MA-MTLN.

MLAGG-Net: Multi-level aggregation and global guidance network for pancreatic lesion segmentation in histopathological images

Automatic multi-tissue segmentation in pancreatic pathological images with selected multi-scale attention network

Multi-target segmentation of pancreas and pancreatic tumor based on fusion of attention mechanism

Lgma-net: liver and tumor segmentation methods based on local–global feature mergence and attention mechanisms

GLGFormer: Global Local Guidance Network for Mucosal Lesion Segmentation in Gastrointestinal Endoscopy Images

Attention-enhanced multiscale feature fusion network for pancreas and tumor segmentation

BAGNet: Bidirectional Aware Guidance Network for Malignant Breast lesions Segmentation

Segmentation of pancreatic tumors based on multi‐scale convolution and channel attention mechanism in the encoder‐decoder scheme

Multi-Scale Context-Guided Deep Network for Automated Lesion Segmentation With Endoscopy Images of Gastrointestinal Tract

Ssm-Net: Semi-Supervised Multi-Task Network for Joint Lesion Segmentation and Classification from Pancreatic Eus Images

Pancreatic cancer pathology image segmentation with channel and spatial long-range dependencies

A three-path network with multi-scale selective feature fusion, edge-inspiring and edge-guiding for liver tumor segmentation

Robust Pancreatic Ductal Adenocarcinoma Segmentation with Multi-institutional Multi-phase Partially-Annotated CT Scans

Multiscale and multiperception feature learning for pancreatic lesion detection based on noncontrast CT

GFA-Net: Global Feature Aggregation Network Based on Contrastive Learning for Breast Lesion Automated Segmentation in Ultrasound Images

LMA-Net: A lesion morphology aware network for medical image segmentation towards breast tumors

SCPMan: Shape context and prior constrained multi-scale attention network for pancreatic segmentation

Dpanet: A Novel Network Based On Dense Pyramid Feature Extractor And Dual Correlation Analysis Attention Modules For Colon Glands Segmentation

3D Multi-Attention Guided Multi-Task Learning Network for Automatic Gastric Tumor Segmentation and Lymph Node Classification.

MG-Net: Multi-level global-aware network for thymoma segmentation

Pancreatic cancer segmentation in unregistered multi-parametric MRI with adversarial learning and multi-scale supervision