Abstract:Multi-Task Learning (MTL) is a framework, where multiple related tasks are learned jointly and benefit from a shared representation space, or parameter transfer. To provide sufficient learning support, modern MTL uses annotated data with full, or sufficiently large overlap across tasks, i.e., each input sample is annotated for all, or most of the tasks. However, collecting such annotations is prohibitive in many real applications, and cannot benefit from datasets available for individual tasks. In this work, we challenge this setup and show that MTL can be successful with classification tasks with little, or non-overlapping annotations, or when there is big discrepancy in the size of labeled data per task. We explore task-relatedness for co-annotation and co-training, and propose a novel approach, where knowledge exchange is enabled between the tasks via distribution matching. To demonstrate the general applicability of our method, we conducted diverse case studies in the domains of affective computing, face recognition, species recognition, and shopping item classification using nine datasets. Our large-scale study of affective tasks for basic expression recognition and facial action unit detection illustrates that our approach is network agnostic and brings large performance improvements compared to the state-of-the-art in both tasks and across all studied databases. In all case studies, we show that co-training via task-relatedness is advantageous and prevents negative transfer (which occurs when MT model's performance is worse than that of at least one single-task model).

Multitask Learning Over Graphs: An Approach for Distributed, Streaming Machine Learning

Online Distributed Learning over Graphs with Multitask Graph-Filter Models

Distributed Learning of Predictive Structures from Multiple Tasks over Networks

Distributed Stochastic Multi-Task Learning with Graph Regularization

Federated Multi-task Graph Learning

Learning over Multitask Graphs -- Part II: Performance Analysis

Distributed Jointly Sparse Multitask Learning over Networks

Multi-task Vector Field Learning.

Distributed Multi-Task Learning with Shared Representation

Distributed Networked Multi-task Learning

Traffic Flow and Speed Forecasting Through a Bayesian Deep Multi-Linear Relationship Network.

Multiple Task Learning Using Iteratively Reweighted Least Square.

Multi-task Learning over Graph Structures.

Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyond

Multi-Task Learning with Group-Specific Feature Space Sharing

Diffusion LMS for multitask problems with overlapping hypothesis subspaces

Online Parallel Multi-Task Relationship Learning via Alternating Direction Method of Multipliers

Task Grouping for Automated Multi-Task Machine Learning via Task Affinity Prediction

Personalized Decentralized Multi-Task Learning Over Dynamic Communication Graphs

Learning Functions to Study the Benefit of Multitask Learning