Abstract:In this paper, we propose a novel efficient multi-task learning formulation for the class of progression problems in which its state will continuously change over time. To use the shared knowledge information between multiple tasks to improve performance, existing multi-task learning methods mainly focus on feature selection or optimizing the task relation structure. The feature selection methods usually fail to explore the complex relationship between tasks and thus have limited performance. The methods centring on optimizing the relation structure of tasks are not capable of selecting meaningful features and have a bi-convex objective function which results in high computation complexity of the associated optimization algorithm. Unlike these multi-task learning methods, motivated by a simple and direct idea that the state of a system at the current time point should be related to all previous time points, we first propose a novel relation structure, termed adaptive global temporal relation structure (AGTS). Then we integrate the widely used sparse group Lasso, fused Lasso with AGTS to propose a novel convex multi-task learning formulation that not only performs feature selection but also adaptively captures the global temporal task relatedness. Since the existence of three non-smooth penalties, the objective function is challenging to solve. We first design an optimization algorithm based on the alternating direction method of multipliers (ADMM). Considering that the worst-case convergence rate of ADMM is only sub-linear, we then devise an efficient algorithm based on the accelerated gradient method which has the optimal convergence rate among first-order methods. We show the proximal operator of several non-smooth penalties can be solved efficiently due to the special structure of our formulation. Experimental results on four real-world datasets demonstrate that our approach not only outperforms multiple baseline MTL methods in terms of effectiveness but also has high efficiency.

Encoding Sparse and Competitive Structures among Tasks in Multi-Task Learning

Multi-task Sparse Structure Learning

Encoding Tree Sparsity in Multi-Task Learning: A Probabilistic Framework.

Multiple Task Learning with Flexible Structure Regularization.

Multiple Task Learning Using Iteratively Reweighted Least Square.

Robust Estimator Based Adaptive Multi-Task Learning

Robust Online Multi-Task Learning with Correlative and Personalized Structures

Latent Group Structured Multi-task Learning

Sequential Cooperative Distillation for Imbalanced Multi-Task Learning

ESSR: Evolving Sparse Sharing Representation for Multi-task Learning

Giving each task what it needs -- leveraging structured sparsity for tailored multi-task learning

Multi-Task Cooperative Learning via Searching for Flat Minima

MSSM: A Multiple-level Sparse Sharing Model for Efficient Multi-Task Learning

Robust Analysis of Multi-Task Learning Efficiency: New Benchmarks on Light-Weighed Backbones and Effective Measurement of Multi-Task Learning Challenges by Feature Disentanglement

Elaborate Multi-Task Subspace Learning with Discrete Group Constraint

Equitable Multi-task Learning

Multilinear Multitask Learning by Rank-Product Regularization.

Low-Rank Multitask Learning based on Tensorized SVMs and LSSVMs

Efficient Multi-Task Learning with Adaptive Temporal Structure for Progression Prediction

On Better Exploring and Exploiting Task Relationships in Multitask Learning: Joint Model and Feature Learning.

Modeling Output-Level Task Relatedness in Multi-Task Learning with Feedback Mechanism