Abstract:Multi-task learning is a successful machine learning framework which improves the performance of prediction models by leveraging knowledge among tasks, e.g., the relationships between different tasks. Most of existing multi-task learning methods focus on guiding learning process by predefined task relationships. In fact, these methods have not fully exploited the associated relationships during the learning process. On the one hand, replacing predefined task relationships by adaptively learned ones may result in higher prediction accuracy as it can avoid the risk of misguiding caused by improperly predefined relationships. On the other hand, apart from the task relationships, feature-task dependence and feature-feature interactions could also be employed to guide the learning process. Along this line, we propose aMultiple Relational Attention Network (MRAN) framework for multi-task learning, in which three types of relationships are considered. Correspondingly, MRAN consists of three attention-based relationship learning modules: 1) a task-task relationship learning module which captures the relationships among tasks automatically and controls the positive and negative knowledge transfer adaptively; 2) a feature-feature interaction learning module that handles the complicated interactions among features; 3) a task-feature dependence learning module, which can associate the related features with target tasks separately. To evaluate the effectiveness of the proposed MARN, experiments are conducted on two public datasets and a real-world dataset crawled from a review hosting site. Experimental results demonstrate the superiority of our method over both classical and the state-of-the-art multi-task learning methods.

Learning multiple tasks with multilinear relationship networks

Learning Multiple Tasks with Multilinear Relationship Networks.

Multiple Relational Attention Network for Multi-task Learning

Traffic Flow and Speed Forecasting Through a Bayesian Deep Multi-Linear Relationship Network.

Multiple Relational Learning Network for Joint Referring Expression Comprehension and Segmentation

Multi-Target Regression Via Robust Low-Rank Learning.

Grouped Multi-Task Learning with Hidden Tasks Enhancement

On Better Exploring and Exploiting Task Relationships in Multitask Learning: Joint Model and Feature Learning.

Learning Task Relational Structure for Multi-task Feature Learning

MRLN: Multi-Task Relational Learning Network for MRI Vertebral Localization, Identification, and Segmentation

Low-Rank Deep Convolutional Neural Network for Multi-Task Learning

Multi-Task Networks With Universe, Group, and Task Feature Learning

Multi-task Model and Feature Joint Learning

Saliency-Regularized Deep Multi-Task Learning

Multi-task Learning with Deformable Convolution

Multi-task Vector Field Learning.

DMTMV: A Unified Learning Framework for Deep Multi-task Multi-view Learning.

Multi-task and Multi-View Training for End-to-end Relation Extraction

Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks

Dynamic Multi-Task Learning with Convolutional Neural Network

Multi-Task Network Representation Learning