Abstract:Automatic gastric tumor segmentation and lymph node (LN) classification not only can assist radiologists in reading images, but also provide image-guided clinical diagnosis and improve diagnosis accuracy. However, due to the inhomogeneous intensity distribution of gastric tumor and LN in CT scans, the ambiguous/missing boundaries, and highly variable shapes of gastric tumor, it is quite challenging to develop an automatic solution. To comprehensively address these challenges, we propose a novel 3D multi-attention guided multi-task learning network for simultaneous gastric tumor segmentation and LN classification, which makes full use of the complementary information extracted from different dimensions, scales, and tasks. Specifically, we tackle task correlation and heterogeneity with the convolutional neural network consisting of scale-aware attention-guided shared feature learning for refined and universal multi-scale features, and task-aware attention-guided feature learning for task-specific discriminative features. This shared feature learning is equipped with two types of scale-aware attention (visual attention and adaptive spatial attention) and two stage-wise deep supervision paths. The task-aware attention-guided feature learning comprises a segmentation-aware attention module and a classification-aware attention module. The proposed 3D multi-task learning network can balance all tasks by combining segmentation and classification loss functions with weight uncertainty. We evaluate our model on an in-house CT images dataset collected from three medical centers. Experimental results demonstrate that our method outperforms the state-of-the-art algorithms, and obtains promising performance for tumor segmentation and LN classification. Moreover, to explore the generalization for other segmentation tasks, we also extend the proposed network to liver tumor segmentation in CT images of the MICCAI 2017 Liver Tumor Segmentation Challenge. Our implementation is released at https://github.com/infinite-tao/MA-MTLN.

Cross-Task Attention Network: Improving Multi-Task Learning for Medical Imaging Applications

MTANet: Multi-Task Attention Network for Automatic Medical Image Segmentation and Classification

Using Multi-Task Learning to Improve Diagnostic Performance of Convolutional Neural Networks

Enhancing CT Image synthesis from multi-modal MRI data based on a multi-task neural network framework

Attention-Enhanced Cross-Task Network for Analysing Multiple Attributes of Lung Nodules in CT

3D Multi-Attention Guided Multi-Task Learning Network for Automatic Gastric Tumor Segmentation and Lymph Node Classification.

Multi-task deep learning for medical image computing and analysis: A review

Simulating cross‐modal medical images using multi‐task adversarial learning of a deep convolutional neural network

Cross-Task Feedback Fusion GAN for Joint MR-CT Synthesis and Segmentation of Target and Organs-at-Risk

Multi-Task Learning for Integrated Automated Contouring and Voxel-Based Dose Prediction in Radiotherapy

Enhanced NSCLC Subtyping and Staging Through Attention-Augmented Multi-Task Deep Learning: A Novel Diagnostic Tool

C2MA-Net: Cross-Modal Cross-Attention Network for Acute Ischemic Stroke Lesion Segmentation Based on CT Perfusion Scans

One-Pass Multi-Task Networks with Cross-Task Guided Attention for Brain Tumor Segmentation

Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentation

Multi-scale Feature Enhancement in Multi-task Learning for Medical Image Analysis

MCHNet: an Efficient Cross Attention-guided Hierarchical Multi-scale Network for Segmentation of Organs at Risk in CT Images

Asymmetric multi-task attention network for prostate bed segmentation in computed tomography images

DenseMTL: Cross-task Attention Mechanism for Dense Multi-task Learning

Cross-modal Attention for MRI and Ultrasound Volume Registration

Deep Cascaded Attention Network for Multi-task Brain Tumor Segmentation.

MCANet: Medical Image Segmentation with Multi-Scale Cross-Axis Attention