Abstract:Multi-task learning (MTL) aims to improve the performance of multiple related tasks by exploiting the intrinsic relationships among them. Recently, multi-task feature learning algorithms have received increasing attention and they have been successfully applied to many applications involving high-dimensional data. However, they assume that all tasks share a common set of features, which is too restrictive and may not hold in real-world applications, since outlier tasks often exist. In this paper, we propose a Robust MultiTask Feature Learning algorithm (rMTFL) which simultaneously captures a common set of features among relevant tasks and identifies outlier tasks. Specifically, we decompose the weight (model) matrix for all tasks into two components. We impose the well-known group Lasso penalty on row groups of the first component for capturing the shared features among relevant tasks. To simultaneously identify the outlier tasks, we impose the same group Lasso penalty but on column groups of the second component. We propose to employ the accelerated gradient descent to efficiently solve the optimization problem in rMTFL, and show that the proposed algorithm is scalable to large-size problems. In addition, we provide a detailed theoretical analysis on the proposed rMTFL formulation. Specifically, we present a theoretical bound to measure how well our proposed rMTFL approximates the true evaluation, and provide bounds to measure the error between the estimated weights of rMTFL and the underlying true weights. Moreover, by assuming that the underlying true weights are above the noise level, we present a sound theoretical result to show how to obtain the underlying true shared features and outlier tasks (sparsity patterns). Empirical studies on both synthetic and real-world data demonstrate that our proposed rMTFL is capable of simultaneously capturing shared features among tasks and identifying outlier tasks.

Robust multitask learning in high dimensions under memory constraint

Robust Estimation and Shrinkage in Ultrahigh Dimensional Expectile Regression with Heavy Tails and Variance Heterogeneity

Distributed Bootstrap Simultaneous Inference for High-Dimensional Quantile Regression

Multitask Learning and Bandits via Robust Statistics

Multi-Target Regression Via Robust Low-Rank Learning.

Distributed Jointly Sparse Multitask Learning over Networks

Optimal Multitask Linear Regression and Contextual Bandits under Sparse Heterogeneity

Variable Selection and Task Grouping for Multi-Task Learning

Learning Incoherent Sparse and Low-Rank Patterns from Multiple Tasks

Proximal Multitask Learning Over Distributed Networks With Jointly Sparse Structure

Distributed Learning of Predictive Structures from Multiple Tasks over Networks

Robust Estimator Based Adaptive Multi-Task Learning

Spectral Algorithm for Low-rank Multitask Regression

Simultaneous Dimension Reduction and Variable Selection for Multinomial Logistic Regression

Modeling Alzheimer's disease cognitive scores using multi-task sparse group lasso

Robust Multi-Task Feature Learning

A Tuning-free Robust and Efficient Approach to High-dimensional Regression

Multitask Multiclass Support Vector Machines: Model and Experiments

Multitask Learning with No Regret: from Improved Confidence Bounds to Active Learning

Robust Multi-Task Learning With Flexible Manifold Constraint

Sparse coding for multitask and transfer learning