Abstract:In this paper, we propose a novel efficient multi-task learning formulation for the class of progression problems in which its state will continuously change over time. To use the shared knowledge information between multiple tasks to improve performance, existing multi-task learning methods mainly focus on feature selection or optimizing the task relation structure. The feature selection methods usually fail to explore the complex relationship between tasks and thus have limited performance. The methods centring on optimizing the relation structure of tasks are not capable of selecting meaningful features and have a bi-convex objective function which results in high computation complexity of the associated optimization algorithm. Unlike these multi-task learning methods, motivated by a simple and direct idea that the state of a system at the current time point should be related to all previous time points, we first propose a novel relation structure, termed adaptive global temporal relation structure (AGTS). Then we integrate the widely used sparse group Lasso, fused Lasso with AGTS to propose a novel convex multi-task learning formulation that not only performs feature selection but also adaptively captures the global temporal task relatedness. Since the existence of three non-smooth penalties, the objective function is challenging to solve. We first design an optimization algorithm based on the alternating direction method of multipliers (ADMM). Considering that the worst-case convergence rate of ADMM is only sub-linear, we then devise an efficient algorithm based on the accelerated gradient method which has the optimal convergence rate among first-order methods. We show the proximal operator of several non-smooth penalties can be solved efficiently due to the special structure of our formulation. Experimental results on four real-world datasets demonstrate that our approach not only outperforms multiple baseline MTL methods in terms of effectiveness but also has high efficiency.

AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning

AdaMerging: Adaptive Model Merging for Multi-Task Learning

Robust Estimator Based Adaptive Multi-Task Learning

Multi-task Gradient Descent for Multi-Task Learning

A Model-Agnostic Approach to Mitigate Gradient Interference for Multi-Task Learning

Towards Impartial Multi-task Learning.

AdapMTL: Adaptive Pruning Framework for Multitask Learning Model

ADMET Property Prediction Via Multi-Task Graph Learning under Adaptive Auxiliary Task Selection

Task Grouping for Automated Multi-Task Machine Learning via Task Affinity Prediction

Fair Resource Allocation in Multi-Task Learning

Efficient Multi-Task Learning with Adaptive Temporal Structure for Progression Prediction

AdaTT: Adaptive Task-to-Task Fusion Network for Multitask Learning in Recommendations

Towards Task-Conflicts Momentum-Calibrated Approach for Multi-task Learning

No More Tuning: Prioritized Multi-Task Learning with Lagrangian Differential Multiplier Methods

An Adaptive Node Network Based on Multitask Deep Learning

Equitable Multi-task Learning

Multiple Task Learning Using Iteratively Reweighted Least Square.

Dual-Balancing for Multi-Task Learning

Adaptive and Robust Multi-Task Learning

AdaMTL: Adaptive Input-dependent Inference for Efficient Multi-Task Learning

Task Variance Regularized Multi-Task Learning.