Abstract:Developing efficient solutions for inference problems in intelligent sensor networks is crucial for the next generation of location, tracking, and mapping services. This paper develops a scalable distributed probabilistic inference algorithm that applies to continuous variables, intractable posteriors and large-scale real-time data in sensor networks. In a centralized setting, variational inference is a fundamental technique for performing approximate Bayesian estimation, in which an intractable posterior density is approximated with a parametric density. Our key contribution lies in the derivation of a separable lower bound on the centralized estimation objective, which enables distributed variational inference with one-hop communication in a sensor network. Our distributed evidence lower bound (DELBO) consists of a weighted sum of observation likelihood and divergence to prior densities, and its gap to the measurement evidence is due to consensus and modeling errors. To solve binary classification and regression problems while handling streaming data, we design an online distributed algorithm that maximizes DELBO, and specialize it to Gaussian variational densities with non-linear likelihoods. The resulting distributed Gaussian variational inference (DGVI) efficiently inverts a $1$-rank correction to the covariance matrix. Finally, we derive a diagonalized version for online distributed inference in high-dimensional models, and apply it to multi-robot probabilistic mapping using indoor LiDAR data.

Incremental Variational Inference for Latent Dirichlet Allocation

Auto-Encoding Variational Bayes

Sparse Hybrid Variational-Gibbs Algorithm for Latent Dirichlet Allocation.

Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsal

Variational Autoencoders for Efficient Simulation-Based Inference

Distributed Variational Inference for Online Supervised Learning

Stochastic variational inference for large-scale discrete choice models using adaptive batch sizes

A Deterministic Global Optimization Method for Variational Inference

LDC-VAE: A Latent Distribution Consistency Approach to Variational AutoEncoders

LaDDer: Latent Data Distribution Modelling with a Generative Prior

Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors

Amortized Variational Inference: When and Why?

Variational inference for the latent shrinkage position model

Variational Inference for Uncertainty on the Inputs of Gaussian Process Models

Automatic Variational Inference in Stan

Recursive Monte Carlo and Variational Inference with Auxiliary Variables

Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Sampling

Variational mcmc

Meta-Amortized Variational Inference and Learning

Variational Gibbs Inference for Statistical Model Estimation from Incomplete Data