Abstract:Multi-task learning (MTL) aims to improve the performance of multiple related tasks by exploiting the intrinsic relationships among them. Recently, multi-task feature learning algorithms have received increasing attention and they have been successfully applied to many applications involving high-dimensional data. However, they assume that all tasks share a common set of features, which is too restrictive and may not hold in real-world applications, since outlier tasks often exist. In this paper, we propose a Robust MultiTask Feature Learning algorithm (rMTFL) which simultaneously captures a common set of features among relevant tasks and identifies outlier tasks. Specifically, we decompose the weight (model) matrix for all tasks into two components. We impose the well-known group Lasso penalty on row groups of the first component for capturing the shared features among relevant tasks. To simultaneously identify the outlier tasks, we impose the same group Lasso penalty but on column groups of the second component. We propose to employ the accelerated gradient descent to efficiently solve the optimization problem in rMTFL, and show that the proposed algorithm is scalable to large-size problems. In addition, we provide a detailed theoretical analysis on the proposed rMTFL formulation. Specifically, we present a theoretical bound to measure how well our proposed rMTFL approximates the true evaluation, and provide bounds to measure the error between the estimated weights of rMTFL and the underlying true weights. Moreover, by assuming that the underlying true weights are above the noise level, we present a sound theoretical result to show how to obtain the underlying true shared features and outlier tasks (sparsity patterns). Empirical studies on both synthetic and real-world data demonstrate that our proposed rMTFL is capable of simultaneously capturing shared features among tasks and identifying outlier tasks.

Task Uncertainty Loss Reduce Negative Transfer in Asymmetric Multi-task Feature Learning

Deep Asymmetric Multi-task Feature Learning

Analytical Uncertainty-Based Loss Weighting in Multi-Task Learning

Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction

EMT-NAS: Transferring Architectural Knowledge Between Tasks from Different Datasets

Loss-Balanced Task Weighting to Reduce Negative Transfer in Multi-Task Learning

Learning Boost by Exploiting the Auxiliary Task in Multi-task Domain

Interpetable Target-Feature Aggregation for Multi-Task Learning based on Bias-Variance Analysis

Robust Multi-Task Feature Learning

Robust Estimator Based Adaptive Multi-Task Learning

A Model-Agnostic Approach to Mitigate Gradient Interference for Multi-Task Learning

Exploiting High-Order Information in Heterogeneous Multi-Task Feature Learning.

Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing with Non-Learnable Primitives

Enabling Asymmetric Knowledge Transfer in Multi-Task Learning with Self-Auxiliaries

ESSR: Evolving Sparse Sharing Representation for Multi-task Learning

Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data

Towards Impartial Multi-task Learning.

Equitable Multi-task Learning

Dual-Balancing for Multi-Task Learning

FairBranch: Mitigating Bias Transfer in Fair Multi-task Learning

Semantic Feature Learning for Heterogeneous Multitask Classification Via Non-Negative Matrix Factorization.