Abstract:Automatic gastric tumor segmentation and lymph node (LN) classification not only can assist radiologists in reading images, but also provide image-guided clinical diagnosis and improve diagnosis accuracy. However, due to the inhomogeneous intensity distribution of gastric tumor and LN in CT scans, the ambiguous/missing boundaries, and highly variable shapes of gastric tumor, it is quite challenging to develop an automatic solution. To comprehensively address these challenges, we propose a novel 3D multi-attention guided multi-task learning network for simultaneous gastric tumor segmentation and LN classification, which makes full use of the complementary information extracted from different dimensions, scales, and tasks. Specifically, we tackle task correlation and heterogeneity with the convolutional neural network consisting of scale-aware attention-guided shared feature learning for refined and universal multi-scale features, and task-aware attention-guided feature learning for task-specific discriminative features. This shared feature learning is equipped with two types of scale-aware attention (visual attention and adaptive spatial attention) and two stage-wise deep supervision paths. The task-aware attention-guided feature learning comprises a segmentation-aware attention module and a classification-aware attention module. The proposed 3D multi-task learning network can balance all tasks by combining segmentation and classification loss functions with weight uncertainty. We evaluate our model on an in-house CT images dataset collected from three medical centers. Experimental results demonstrate that our method outperforms the state-of-the-art algorithms, and obtains promising performance for tumor segmentation and LN classification. Moreover, to explore the generalization for other segmentation tasks, we also extend the proposed network to liver tumor segmentation in CT images of the MICCAI 2017 Liver Tumor Segmentation Challenge. Our implementation is released at https://github.com/infinite-tao/MA-MTLN.

Multi-task learning based on geometric invariance discriminative features

Component Identification and Depth Estimation for Structural Images Based on Multi-Scale Task Interaction Network

DenseMTL: Cross-task Attention Mechanism for Dense Multi-task Learning

Multi-task Vector Field Learning.

DeMT: Deformable Mixer Transformer for Multi-Task Learning of Dense Prediction

Deformable Mixer Transformer with Gating for Multi-Task Learning of Dense Prediction

SGW-based Multi-Task Learning in Vision Tasks

Deep Heterogeneous Multi-Task Metric Learning for Visual Recognition and Retrieval

Cross-Task Affinity Learning for Multitask Dense Scene Predictions

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

Multi-scale Feature Enhancement in Multi-task Learning for Medical Image Analysis

Constructing Multiple Tasks for Augmentation: Improving Neural Image Classification with K-means Features

Multi-task Model and Feature Joint Learning

Using Multi-Task Learning to Improve Diagnostic Performance of Convolutional Neural Networks

MulT: An End-to-End Multitask Learning Transformer

3D Multi-Attention Guided Multi-Task Learning Network for Automatic Gastric Tumor Segmentation and Lymph Node Classification.

Robust Analysis of Multi-Task Learning Efficiency: New Benchmarks on Light-Weighed Backbones and Effective Measurement of Multi-Task Learning Challenges by Feature Disentanglement

HirMTL: Hierarchical Multi-Task Learning for dense scene understanding

Optimizing Dense Visual Predictions Through Multi-Task Coherence and Prioritization

Exploring multi-tasking learning in document attribute classification

STG-MTL: Scalable Task Grouping for Multi-Task Learning Using Data Map