Abstract:Multitask learning (MTL) is a joint learning paradigm, which fuses multiple related tasks together to achieve the better performance than single-task learning methods. It has been observed by many researchers that different tasks with certain similarities share a low-dimensional common yet latent subspace. In order to get the low-rank structure shared across tasks, trace norm has been used as a convex relaxation of the rank minimization problem. However, trace norm is not a tight approximation for the rank function. To address this important issue, we propose two novel regularization-based models to approximate the rank minimization problem by minimizing the k minimal singular values. For our new models, if the minimal singular values are suppressed to zeros, the rank would also be reduced. Compared with the standard trace norm, our new regularization-based models are the tighter approximations, which can help our models capture the low-dimensional subspace among multiple tasks better. Besides, it is an NP-hard problem to directly solve the exact rank minimization problem for our models. In this article, we proposed two simple but effective strategies to optimize our models, which tactically solves the exact rank minimization problem by setting a large penalizing parameter. Experimental results performed on synthetic and real-world benchmark datasets demonstrate that the proposed models have the ability of learning the low-rank structure shared across tasks and the better performance than other classical MTL methods.

Calibrated Multi-Task Learning

Calibrated Multi-Task Subspace Learning Via Binary Group Structure Constraint.

Multiple Task Learning Using Iteratively Reweighted Least Square.

Robust Estimator Based Adaptive Multi-Task Learning

Low-Rank Deep Convolutional Neural Network for Multi-Task Learning

Multi-Task Learning Regression via Convex Clustering

Multiple Task Learning with Flexible Structure Regularization.

Multi-task Model and Feature Joint Learning

Multi-task Vector Field Learning.

Learning Incoherent Sparse and Low-Rank Patterns from Multiple Tasks

Multilinear Multitask Learning by Rank-Product Regularization.

Robust Online Multi-Task Learning with Correlative and Personalized Structures

Transductive Matrix Completion with Calibration for Multi-Task Learning

Multi-Stage Multi-Task Feature Learning

Multi-task learning via robust regularized clustering with non-convex group penalties

Multitask Learning for Classification Problem Via New Tight Relaxation of Rank Minimization.

Convex Multitask Learning with Flexible Task Clusters

Self-Paced Multi-Task Learning

New Tight Relaxations of Rank Minimization for Multi-Task Learning.

Calibrated Multivariate Regression Networks

Multi-Task Clustering with Model Relation Learning.