Abstract:In this paper, we propose a novel efficient multi-task learning formulation for the class of progression problems in which its state will continuously change over time. To use the shared knowledge information between multiple tasks to improve performance, existing multi-task learning methods mainly focus on feature selection or optimizing the task relation structure. The feature selection methods usually fail to explore the complex relationship between tasks and thus have limited performance. The methods centring on optimizing the relation structure of tasks are not capable of selecting meaningful features and have a bi-convex objective function which results in high computation complexity of the associated optimization algorithm. Unlike these multi-task learning methods, motivated by a simple and direct idea that the state of a system at the current time point should be related to all previous time points, we first propose a novel relation structure, termed adaptive global temporal relation structure (AGTS). Then we integrate the widely used sparse group Lasso, fused Lasso with AGTS to propose a novel convex multi-task learning formulation that not only performs feature selection but also adaptively captures the global temporal task relatedness. Since the existence of three non-smooth penalties, the objective function is challenging to solve. We first design an optimization algorithm based on the alternating direction method of multipliers (ADMM). Considering that the worst-case convergence rate of ADMM is only sub-linear, we then devise an efficient algorithm based on the accelerated gradient method which has the optimal convergence rate among first-order methods. We show the proximal operator of several non-smooth penalties can be solved efficiently due to the special structure of our formulation. Experimental results on four real-world datasets demonstrate that our approach not only outperforms multiple baseline MTL methods in terms of effectiveness but also has high efficiency.

Online Parallel Multi-Task Relationship Learning via Alternating Direction Method of Multipliers

Multitask Diffusion Adaptation over Networks with Common Latent Representations

GDOD: Effective Gradient Descent using Orthogonal Decomposition for Multi-Task Learning

Zeroth-Order Online Alternating Direction Method of Multipliers: Convergence Analysis and Applications.

Online Learning of Multiple Tasks and Their Relationships : Testing on Spam Email Data and EEG Signals Recorded in Construction Fields

A Fast Alternating Direction Method of Multipliers Algorithm for Big Data Applications.

Efficient Multi-Task Learning with Adaptive Temporal Structure for Progression Prediction

Online Learning over a Decentralized Network Through ADMM

Distributed Multi-Task Relationship Learning

Online Federated Multitask Learning.

An Adaptive Low Computational Cost Alternating Direction Method of Multiplier for RELM Large-Scale Distributed Optimization

OMS-TL

Multi-Task Learning as Multi-Objective Optimization

Robust Online Multi-Task Learning with Correlative and Personalized Structures

Domain Adaptation Multitask Optimization

Byzantine Resilient Distributed Multi-Task Learning

Algorithm Design for Online Meta-Learning with Task Boundary Detection

A Multi-Head Ensemble Multi-Task Learning Approach for Dynamical Computation Offloading

Multi-Task Learning with Multi-Task Optimization

A Multiple Gradient Descent Design for Multi-Task Learning on Edge Computing: Multi-Objective Machine Learning Approach

Solving many-task optimization problems via online intertask learning