Abstract:The foundation of multi-task learning lies in the collaboration and interaction among tasks. However, in numerous real-world scenarios, certain tasks usually necessitate distinct, specialized knowledge. The mixing of these different task-specific knowledge often results in gradient conflicts during the optimization process, posing a significant challenge in the design of effective multi-task learning systems. This study proposes a straightforward yet effective multi-task learning framework that employs groups of expert networks to decouple the learning of task-specific knowledge and mitigate such gradient conflicts. Specifically, this approach partitions the feature channels into task-specific and shared components. The task-specific subsets are processed by dedicated experts to distill specialized knowledge. The shared features are captured by a point-wise aggregation layer from the whole outputs of all experts, demonstrating superior performance in capturing inter-task interactions. By considering both task-specific knowledge and shared features, the proposed approach exhibits superior performance in multi-task learning. Extensive experiments conducted on the PASCAL-Context and NYUD-v2 datasets have demonstrated the superiority of the proposed approach compared to other state-of-the-art methods. Furthermore, a benchmark dataset for multi-task learning in underwater scenarios has been developed, encompassing object detection and underwater image enhancement tasks. Comprehensive experiments on this dataset consistently validate the effectiveness of the proposed multi-task learning strategy. The source code is available at https://github.com/chenjie04/Multi-Task-Learning-PyTorch.

Fitting and sharing multi-task learning

Learning Task Grouping and Overlap in Multi-task Learning

Sharing Knowledge in Multi-Task Deep Reinforcement Learning

Learning Sparse Sharing Architectures for Multiple Tasks.

Informative Relationship Multi-Task Learning: Exploring Pairwise Contribution Across Tasks' Sharing Knowledge

ESSR: Evolving Sparse Sharing Representation for Multi-task Learning

MSSM: A Multiple-level Sparse Sharing Model for Efficient Multi-Task Learning

Multi-task Model and Feature Joint Learning

Adaptive Hard Parameter Sharing Method Based on Multi-Task Deep Learning

Efficiently Identifying Task Groupings for Multi-Task Learning

Adaptively Sharing Multi-Levels of Distributed Representations in Multi-Task Learning

Adaptive and Robust Multi-Task Learning

Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction

Adaptive Sharing for Image Classification.

Task's Choice: Pruning-Based Feature Sharing (PBFS) for Multi-Task Learning.

Mitigating Gradient Conflicts Via Expert Squads in Multi-Task Learning

Multi-Task Learning with Group-Specific Feature Space Sharing

Towards Impartial Multi-task Learning.

Robust Estimator Based Adaptive Multi-Task Learning

Exploiting Task-Feature Co-Clusters In Multi-Task Learning

Multi-Task Learning Via Sharing Inexact Low-Rank Subspace