Abstract:Multi-task learning (MTL) aims to improve estimation and prediction performance by sharing common information among related tasks. One natural assumption in MTL is that tasks are classified into clusters based on their characteristics. However, existing MTL methods based on this assumption often ignore outlier tasks that have large task-specific components or no relation to other tasks. To address this issue, we propose a novel MTL method called Multi-Task Learning via Robust Regularized Clustering (MTLRRC). MTLRRC incorporates robust regularization terms inspired by robust convex clustering, which is further extended to handle non-convex and group-sparse penalties. The extension allows MTLRRC to simultaneously perform robust task clustering and outlier task detection. The connection between the extended robust clustering and the multivariate M-estimator is also established. This provides an interpretation of the robustness of MTLRRC against outlier tasks. An efficient algorithm based on a modified alternating direction method of multipliers is developed for the estimation of the parameters. The effectiveness of MTLRRC is demonstrated through simulation studies and application to real data.

What problem does this paper attempt to address?

The paper primarily addresses the issue of task clustering in Multi-task Learning (MTL) and specifically focuses on how to handle outlier tasks, which are tasks that are either not closely related to other tasks or have significant specific task characteristics. The paper proposes a new method called Multi-Task Learning via Robust Regularized Clustering (MTLRRC). Specifically, the problems addressed in the paper can be summarized as follows: 1. **Problems with existing MTL methods**: - Existing MTL methods usually assume that tasks can be classified into different clusters based on their characteristics, but these methods often ignore the presence of outlier tasks. - When outlier tasks are present, traditional MTL methods may lead to inaccurate estimation results because they attempt to fit all tasks into certain clusters and make each task's model close to the cluster center, which may result in misinterpretation of outlier tasks. 2. **Proposed solution**: - The paper proposes the MTLRRC method, which combines a loss function and a robust regularization term to perform task clustering and outlier task detection simultaneously. - MTLRRC is inspired by robust convex clustering and further extends it to handle non-convex and group sparse penalties. This approach allows MTLRRC to perform robust task clustering and outlier task detection simultaneously. - The paper also establishes a connection between extended robust clustering and multivariate M-estimators, providing an intuitive explanation for MTLRRC's robustness to outlier tasks. - To estimate the parameters in MTLRRC, an efficient algorithm based on a modified Alternating Direction Method of Multipliers (ADMM) is developed. 3. **Theoretical contributions**: - The paper extends robust convex clustering by introducing non-convex and group sparse penalties and demonstrates the connection between the extended robust clustering problem and multivariate M-estimators. - These theoretical contributions provide a foundation for understanding the robustness of the MTLRRC method and help explain how it effectively identifies outlier tasks and achieves robust task clustering. In summary, the main goal of this paper is to improve the accuracy and robustness of task clustering in multi-task learning scenarios when outlier tasks are present. By introducing the MTLRRC method, the authors propose a new framework to address these issues and demonstrate its effectiveness in both simulation studies and real datasets.

Multi-task learning via robust regularized clustering with non-convex group penalties

Multi-Task Learning Regression via Convex Clustering

Robust Estimator Based Adaptive Multi-Task Learning

Robust Multi-Task Feature Learning

Grouped Multi-Task Learning with Hidden Tasks Enhancement

Convex Multitask Learning with Flexible Task Clusters

Robust Online Multi-Task Learning with Correlative and Personalized Structures

Robust Multi-Task Learning With Flexible Manifold Constraint

Multiple Task Learning Using Iteratively Reweighted Least Square.

Multiple Task Learning with Flexible Structure Regularization.

A Divide-and-Conquer Method for Scalable Robust Multitask Learning

Robust Multi-Task Learning with Excess Risks

Multi-Target Regression Via Robust Low-Rank Learning.

Robust Task Grouping with Representative Tasks for Clustered Multi-Task Learning

Multi-Task Clustering with Model Relation Learning.

Multi-Task Learning with Group-Specific Feature Space Sharing

Multilinear Multitask Learning by Rank-Product Regularization.

Multitask Learning Can Improve Worst-Group Outcomes

A Convex Formulation for Learning Task Relationships in Multi-Task Learning

Task-group Relatedness and Generalization Bounds for Regularized Multi-task Learning.

Network Clustering for Multi-task Learning