Abstract:As an effective learning paradigm against insufficient training samples, multi-task learning (MTL) encourages knowledge sharing across multiple related tasks so as to improve the overall performance. In MTL, a major challenge springs from the phenomenon that sharing the knowledge with dissimilar and hard tasks, known as negative transfer, often results in a worsened performance. Though a substantial amount of studies have been carried out against the negative transfer, most of the existing methods only model the transfer relationship as task correlations, with the transfer across features and tasks left unconsidered. Different from the existing methods, our goal is to alleviate negative transfer collaboratively across features and tasks. To this end, we propose a novel multi-task learning method called task-feature collaborative learning (TFCL). Specifically, we first propose a base model with a heterogeneous block-diagonal structure regularizer to leverage the collaborative grouping of features and tasks and suppressing inter-group knowledge sharing. We then propose an optimization method for the model. Extensive theoretical analysis shows that our proposed method has the following benefits: (a) it enjoys the global convergence property and (b) it provides a block-diagonal structure recovery guarantee. As a practical extension, we extend the base model by allowing overlapping features and differentiating the hard tasks. We further apply it to the personalized attribute prediction problem with fine-grained modeling of user behaviors. Finally, experimental results on both simulated dataset and real-world datasets demonstrate the effectiveness of our proposed method.

Network Clustering for Multi-task Learning

A Multi-Task Learning Strategy for Unsupervised Clustering Via Explicitly Separating the Commonality

Robust Estimator Based Adaptive Multi-Task Learning

Efficient Group Learning with Hypergraph Partition in Multi-task Learning.

Semisoft Task Clustering for Multi-Task Learning

An Adaptive Node Network Based on Multitask Deep Learning

Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction

Convex Multitask Learning with Flexible Task Clusters

Multi-Task Learning with Group-Specific Feature Space Sharing

Multi-Task Learning Regression via Convex Clustering

Heterogeneous Multi-task Semantic Feature Learning for Classification

Multi-Task Clustering with Model Relation Learning.

Multi-task Model and Feature Joint Learning

Multi-Task Learning in Natural Language Processing: An Overview

Multiple Task Learning Using Iteratively Reweighted Least Square.

Robust Online Multi-Task Learning with Correlative and Personalized Structures

Multi-task self-supervised learning based fusion representation for Multi-view clustering

Adaptive Hard Parameter Sharing Method Based on Multi-Task Deep Learning

Robust Task Grouping with Representative Tasks for Clustered Multi-Task Learning

HirMTL: Hierarchical Multi-Task Learning for dense scene understanding