Abstract:Learning discriminative task-specific features simultaneously for multiple distinct tasks is a fundamental problem in multi-task learning. Recent state-of-the-art models consider directly decoding task-specific features from one shared task-generic feature (e.g., feature from a backbone layer), and utilize carefully designed decoders to produce multi-task features. However, as the input feature is fully shared and each task decoder also shares decoding parameters for different input samples, it leads to a static feature decoding process, producing less discriminative task-specific representations. To tackle this limitation, we propose TaskExpert, a novel multi-task mixture-of-experts model that enables learning multiple representative task-generic feature spaces and decoding task-specific features in a dynamic manner. Specifically, TaskExpert introduces a set of expert networks to decompose the backbone feature into several representative task-generic features. Then, the task-specific features are decoded by using dynamic task-specific gating networks operating on the decomposed task-generic features. Furthermore, to establish long-range modeling of the task-specific representations from different layers of TaskExpert, we design a multi-task feature memory that updates at each layer and acts as an additional feature expert for dynamic task-specific feature decoding. Extensive experiments demonstrate that our TaskExpert clearly outperforms previous best-performing methods on all 9 metrics of two competitive multi-task learning benchmarks for visual scene understanding (i.e., PASCAL-Context and NYUD-v2). Codes and models will be made publicly available at <a class="link-external link-https" href="https://github.com/prismformore/Multi-Task-Transformer" rel="external noopener nofollow">this https URL</a>

Modeling Task Relationships in Multi-variate Soft Sensor with Balanced Mixture-of-Experts

Modeling Task Relationships in Multivariate Soft Sensor with Balanced Mixture-of-Experts

TMoE-P: Towards the Pareto Optimum for Multivariate Soft Sensors.

Multiple Model Soft Sensor Based on Affinity Propagation, Gaussian Process and Bayesian Committee Machine

Adaptive Soft Sensor Modeling Method Based on Multi-Model Dynamic Fusion and Its Industrial Application.

Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts

Multi-Task Learning with Calibrated Mixture of Insightful Experts

Semisupervised Robust Modeling of Multimode Industrial Processes for Quality Variable Prediction Based on Student's T Mixture Model.

Student's-t Mixture Regression-Based Robust Soft Sensor Development for Multimode Industrial Processes.

Soft Sensor Model Development in Multiphase/multimode Processes Based on Gaussian Mixture Regression

Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging

Loss-Balanced Task Weighting to Reduce Negative Transfer in Multi-Task Learning

Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts

An adaptive multi-scale feature fusion and adaptive mixture-of-experts multi-task model for industrial equipment health status assessment and remaining useful life prediction

Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners

Harder Tasks Need More Experts: Dynamic Routing in MoE Models

MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts

Multi-Head Mixture-of-Experts

TaskExpert: Dynamically Assembling Multi-Task Representations with Memorial Mixture-of-Experts

HMoE: Heterogeneous Mixture of Experts for Language Modeling

Merging Multi-Task Models via Weight-Ensembling Mixture of Experts