Abstract:Distributed multi-task learning provides significant advantages in multi-agent networks with heterogeneous data sources where agents aim to learn distinct but correlated models <a class="link-external link-http" href="http://simultaneously.However" rel="external noopener nofollow">this http URL</a>, distributed algorithms for learning relatedness among tasks are not resilient in the presence of Byzantine agents. In this paper, we present an approach for Byzantine resilient distributed multi-task learning. We propose an efficient online weight assignment rule by measuring the accumulated loss using an agent's data and its neighbors' models. A small accumulated loss indicates a large similarity between the two tasks. In order to ensure the Byzantine resilience of the aggregation at a normal agent, we introduce a step for filtering out larger losses. We analyze the approach for convex models and show that normal agents converge resiliently towards the global <a class="link-external link-http" href="http://minimum.Further" rel="external noopener nofollow">this http URL</a>, aggregation with the proposed weight assignment rule always results in an improved expected regret than the non-cooperative case. Finally, we demonstrate the approach using three case studies, including regression and classification problems, and show that our method exhibits good empirical performance for non-convex models, such as convolutional neural networks.

Distributed Learning of Predictive Structures from Multiple Tasks over Networks

Distributed Jointly Sparse Multitask Learning over Networks

Distributed Networked Multi-task Learning

Diffusion LMS over Multitask Networks.

Distributed TLS over Multitask Networks with Adaptive Intertask Cooperation

Diffusion LMS for Clustered Multitask Networks

Proximal Multitask Learning Over Distributed Networks With Jointly Sparse Structure

Multitask Diffusion Adaptation over Networks with Common Latent Representations

Distributed adaptive clustering learning over time-varying multitask networks

Distributed Multi-Task Relationship Learning

Adaptive Clustering for Multitask Diffusion Networks

Multitask Diffusion Affine Projection Sign Algorithm and Its Sparse Variant for Distributed Estimation

Distributed Multi-Task Learning with Shared Representation

Penalty-Based Multitask Distributed Adaptation over Networks with Constraints

Diffusion Lms Over Multitask Networks With Noisy Links

Adaptively Sharing Multi-Levels of Distributed Representations in Multi-Task Learning

Multitask Diffusion Adaptation over Networks

Exact Subspace Diffusion for Decentralized Multitask Learning

Byzantine Resilient Distributed Multi-Task Learning

Adaptive dual graph regularization for clustered multi-task learning