Abstract:We consider the problem of learning incoherent sparse and low-rank patterns from multiple tasks. Our approach is based on a linear multi-task learning formulation, in which the sparse and low-rank patterns are induced by a cardinality regularization term and a low-rank constraint, respectively. This formulation is non-convex; we convert it into its convex surrogate, which can be routinely solved via semidefinite programming for small-size problems. We propose to employ the general projected gradient scheme to efficiently solve such a convex surrogate; however, in the optimization formulation, the objective function is non-differentiable and the feasible domain is non-trivial. We present the procedures for computing the projected gradient and ensuring the global convergence of the projected gradient scheme. The computation of projected gradient involves a constrained optimization problem; we show that the optimal solution to such a problem can be obtained via solving an unconstrained optimization subproblem and an Euclidean projection subproblem. We also present two projected gradient algorithms and analyze their rates of convergence in details. In addition, we illustrate the use of the presented projected gradient algorithms for the proposed multi-task learning formulation using the least squares loss. Experimental results on a collection of real-world data sets demonstrate the effectiveness of the proposed multi-task learning formulation and the efficiency of the proposed projected gradient algorithms.

Manifold-regularized Multitask Fuzzy System Modeling with Low-rank and Sparse Structures in Consequent Parameters

Multitask TSK Fuzzy System Modeling by Jointly Reducing Rules and Consequent Parameters.

Modeling Task Relationships in Multivariate Soft Sensor with Balanced Mixture-of-Experts

Distributed Learning of Predictive Structures from Multiple Tasks over Networks

Distributed Jointly Sparse Multitask Learning over Networks

Low-Rank Multitask Learning based on Tensorized SVMs and LSSVMs

Robust Multi-Task Learning With Flexible Manifold Constraint

Proximal Multitask Learning Over Distributed Networks With Jointly Sparse Structure

Tensorized LSSVMS for Multitask Regression

Efficient Pareto Manifold Learning with Low-Rank Structure

Learning task structure via sparsity grouped multitask learning

Dual Low-Rank Multimodal Fusion

Flexible Multi-Task Learning with Latent Task Grouping.

On Multiplicative Multitask Feature Learning

Single-task and multitask sparse Gaussian processes

Multilinear Multitask Learning by Rank-Product Regularization.

A Novel Takagi–Sugeno Fuzzy System Modeling Method with Joint Feature Selection and Rule Reduction

Multiple Task Learning with Flexible Structure Regularization.

Joint Block Structure Sparse Representation for Multi-Input–Multi-Output (MIMO) T–S Fuzzy System Identification

Learning Incoherent Sparse and Low-Rank Patterns from Multiple Tasks

Saliency-Regularized Deep Multi-Task Learning