Abstract:Due to the expanding scope of machine learning (ML) to the fields of sensor networking, cooperative robotics and many other multi-agent systems, distributed deployment of inference algorithms has received a lot of attention. These algorithms involve collaboratively learning unknown parameters from dispersed data collected by multiple agents. There are two competing aspects in such algorithms, namely, intra-agent computation and inter-agent communication. Traditionally, algorithms are designed to perform both synchronously. However, certain circumstances need frugal use of communication channels as they are either unreliable, time-consuming, or resource-expensive. In this paper, we propose gossip-based asynchronous communication to leverage fast computations and reduce communication overhead simultaneously. We analyze the effects of multiple (local) intra-agent computations by the active agents between successive inter-agent communications. For local computations, Bayesian sampling via unadjusted Langevin algorithm (ULA) MCMC is utilized. The communication is assumed to be over a connected graph (e.g., as in decentralized learning), however, the results can be extended to coordinated communication where there is a central server (e.g., federated learning). We theoretically quantify the convergence rates in the process. To demonstrate the efficacy of the proposed algorithm, we present simulations on a toy problem as well as on real world data sets to train ML models to perform classification tasks. We observe faster initial convergence and improved performance accuracy, especially in the low data range. We achieve on average 78% and over 90% classification accuracy respectively on the Gamma Telescope and mHealth data sets from the UCI ML repository.

Distributed Bayesian Inference Over Sensor Networks

Distributed Variational Bayesian Algorithms Over Sensor Networks

A Stochastic Particle Variational Bayesian Inference Inspired Deep-Unfolding Network for Sensing over Wireless Networks

Distributed Jointly Sparse Bayesian Learning with Quantized Communication.

Fully Distributed Variational Bayesian Non-Linear Filter with Unknown Measurement Noise in Sensor Networks

A Fast Distributed Variational Bayesian Filtering for Multisensor LTV System With Non-Gaussian Noise

Distributed Robust Bayesian Filtering for State Estimation.

Trust-Region Based Stochastic Variational Inference for Distributed and Asynchronous Networks

Distributed Bayesian Estimation in Sensor Networks: Consensus on Marginal Densities

Distributed Variational Inference for Online Supervised Learning

Distributed Sequential State Estimation Over Binary Sensor Networks With Inaccurate Process Noise Covariance: A Variational Bayesian Framework

Distributed Bayesian Estimation of Linear Models With Unknown Observation Covariances.

Distributed Censored Regression over Networks

Dynamic Variational Bayesian Student's T Mixture Regression With Hidden Variables Propagation for Industrial Inferential Sensor Development

Semi-supervised Variational Bayesian Student’s T Mixture Regression and Robust Inferential Sensor Application

Asynchronous Local Computations in Distributed Bayesian Learning

1-Bit Compressive Diffusion in Networks Based on Bayesian Inference.

Distributed variational Bayesian adaptive filtering for randomly delayed measurements and unknown noise statistics in multi-sensor networked systems

Joint Variational Bayesian Based Localization Estimation Algorithm on Distributed Gas Source Sensor Network

A Stochastic Particle Variational Bayesian Inference Inspired Deep-Unfolding Network for Non-Convex Parameter Estimation

Consensus variational Bayesian moving horizon estimation for distributed sensor networks with unknown noise covariances