Abstract:Decision-makers often simultaneously face many related but heterogeneous learning problems. For instance, a large retailer may wish to learn product demand at different stores to solve pricing or inventory problems, making it desirable to learn jointly for stores serving similar customers; alternatively, a hospital network may wish to learn patient risk at different providers to allocate personalized interventions, making it desirable to learn jointly for hospitals serving similar patient populations. Motivated by real datasets, we study a natural setting where the unknown parameter in each learning instance can be decomposed into a shared global parameter plus a sparse instance-specific term. We propose a novel two-stage multitask learning estimator that exploits this structure in a sample-efficient way, using a unique combination of robust statistics (to learn across similar instances) and LASSO regression (to debias the results). Our estimator yields improved sample complexity bounds in the feature dimension $d$ relative to commonly-employed estimators; this improvement is exponential for "data-poor" instances, which benefit the most from multitask learning. We illustrate the utility of these results for online learning by embedding our multitask estimator within simultaneous contextual bandit algorithms. We specify a dynamic calibration of our estimator to appropriately balance the bias-variance tradeoff over time, improving the resulting regret bounds in the context dimension $d$. Finally, we illustrate the value of our approach on synthetic and real datasets.

A Bayesian approach to multi-task learning with network lasso

A Bayesian Network nearest k-labels method for Multi-label classification

Distributed Learning of Predictive Structures from Multiple Tasks over Networks

Distributed Jointly Sparse Multitask Learning over Networks

Nonparametric Bayesian Multi-task Learning with Max-margin Posterior Regularization

Multitask Learning and Bandits via Robust Statistics

Sparse Overlapping Sets Lasso for Multitask Learning and its Application to fMRI Analysis

Traffic Flow and Speed Forecasting Through a Bayesian Deep Multi-Linear Relationship Network.

Bayesian Fused Lasso Modeling for Binary Data

Bayesian Max-margin Multi-Task Learning with Data Augmentation.

Bayesian Fused Lasso regression for dynamic binary networks

Label-Sensitive Task Grouping by Bayesian Nonparametric Approach for Multi-Task Multi-Label Learning.

Bayesian learning of feature spaces for multitask regression

The joint graphical lasso for inverse covariance estimation across multiple classes

Classifying Partially Labeled Networked Data via Logistic Network Lasso

Bayesian Discovery of Multiple Bayesian Networks via Transfer Learning

Bayesian Joint Spike-and-Slab Graphical Lasso

Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks.

Bayesian Lasso with neighborhood regression method for Gaussian graphical model

Diagnostic Monitoring of High-Dimensional Networked Systems Via a LASSO-BN Formulation

Bayesian Approach to Linear Bayesian Networks