Abstract:Due to the expanding scope of machine learning (ML) to the fields of sensor networking, cooperative robotics and many other multi-agent systems, distributed deployment of inference algorithms has received a lot of attention. These algorithms involve collaboratively learning unknown parameters from dispersed data collected by multiple agents. There are two competing aspects in such algorithms, namely, intra-agent computation and inter-agent communication. Traditionally, algorithms are designed to perform both synchronously. However, certain circumstances need frugal use of communication channels as they are either unreliable, time-consuming, or resource-expensive. In this paper, we propose gossip-based asynchronous communication to leverage fast computations and reduce communication overhead simultaneously. We analyze the effects of multiple (local) intra-agent computations by the active agents between successive inter-agent communications. For local computations, Bayesian sampling via unadjusted Langevin algorithm (ULA) MCMC is utilized. The communication is assumed to be over a connected graph (e.g., as in decentralized learning), however, the results can be extended to coordinated communication where there is a central server (e.g., federated learning). We theoretically quantify the convergence rates in the process. To demonstrate the efficacy of the proposed algorithm, we present simulations on a toy problem as well as on real world data sets to train ML models to perform classification tasks. We observe faster initial convergence and improved performance accuracy, especially in the low data range. We achieve on average 78% and over 90% classification accuracy respectively on the Gamma Telescope and mHealth data sets from the UCI ML repository.

Distributed Hypothesis Testing With Social Learning and Symmetric Fusion

A Novel Decision Fusion Scheme with Feedback in Neyman-Pearson Detection Systems.

Behavioral Utility-Based Distributed Detection With Conditionally Independent Observations

Beliefs in Decision-Making Cascades

Optimal and Suboptimal Distributed Decision Fusion

Decision Making in Star Networks With Incorrect Beliefs

Memory-Aware Social Learning under Partial Information Sharing

Distributed Joint Detection and Estimation: A Sequential Approach

Network Fault-tolerant and Byzantine-resilient Social Learning via Collaborative Hierarchical Non-Bayesian Learning

Partial Information Sharing over Social Learning Networks

Social teaching: Being informative vs. being right in sequential decision making

Detection of Malicious Agents in Social Learning

Distributed Sequential Detection for Gaussian Shift-in-Mean Hypothesis Testing

Distributed Sequential Hypothesis Testing with Byzantine Sensors.

Asynchronous Local Computations in Distributed Bayesian Learning

Human collective intelligence as distributed Bayesian inference

Distributed Detection Fusion via Monte Carlo Importance Sampling

On the Fusion Strategies for Federated Decision Making

Social Opinion Formation and Decision Making Under Communication Trends

Recursive Distributed Detection for Composite Hypothesis Testing: Nonlinear Observation Models in Additive Gaussian Noise