Abstract:In this paper, we propose a novel efficient multi-task learning formulation for the class of progression problems in which its state will continuously change over time. To use the shared knowledge information between multiple tasks to improve performance, existing multi-task learning methods mainly focus on feature selection or optimizing the task relation structure. The feature selection methods usually fail to explore the complex relationship between tasks and thus have limited performance. The methods centring on optimizing the relation structure of tasks are not capable of selecting meaningful features and have a bi-convex objective function which results in high computation complexity of the associated optimization algorithm. Unlike these multi-task learning methods, motivated by a simple and direct idea that the state of a system at the current time point should be related to all previous time points, we first propose a novel relation structure, termed adaptive global temporal relation structure (AGTS). Then we integrate the widely used sparse group Lasso, fused Lasso with AGTS to propose a novel convex multi-task learning formulation that not only performs feature selection but also adaptively captures the global temporal task relatedness. Since the existence of three non-smooth penalties, the objective function is challenging to solve. We first design an optimization algorithm based on the alternating direction method of multipliers (ADMM). Considering that the worst-case convergence rate of ADMM is only sub-linear, we then devise an efficient algorithm based on the accelerated gradient method which has the optimal convergence rate among first-order methods. We show the proximal operator of several non-smooth penalties can be solved efficiently due to the special structure of our formulation. Experimental results on four real-world datasets demonstrate that our approach not only outperforms multiple baseline MTL methods in terms of effectiveness but also has high efficiency.

Robust Estimator Based Adaptive Multi-Task Learning

Robust Multi-Task Feature Learning

Multi-task learning via robust regularized clustering with non-convex group penalties

Robust Online Multi-Task Learning with Correlative and Personalized Structures

Convex Multitask Learning with Flexible Task Clusters

Adaptive and Robust Multi-Task Learning

AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning

Robust Multi-Task Learning with Excess Risks

Multiple Task Learning Using Iteratively Reweighted Least Square.

Multi-Task Learning Regression via Convex Clustering

A Divide-and-Conquer Method for Scalable Robust Multitask Learning

Efficient Multi-Task Learning with Adaptive Temporal Structure for Progression Prediction

Robust Analysis of Multi-Task Learning Efficiency: New Benchmarks on Light-Weighed Backbones and Effective Measurement of Multi-Task Learning Challenges by Feature Disentanglement

Robust Multi-Task Learning With Flexible Manifold Constraint

Multi-task Model and Feature Joint Learning

Equitable Multi-task Learning

Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction

Robust Task Grouping with Representative Tasks for Clustered Multi-Task Learning

Task Grouping for Automated Multi-Task Machine Learning via Task Affinity Prediction

Adaptive Activation Network and Functional Regularization for Efficient and Flexible Deep Multi-Task Learning

An Adaptive Node Network Based on Multitask Deep Learning