Abstract:Stochastic Gradient Langevin Dynamics (SGLD) have been widely used for Bayesian sampling from certain probability distributions, incorporating derivatives of the log-posterior. With the derivative evaluation of the log-posterior distribution, SGLD methods generate samples from the distribution through performing as a thermostats dynamics that traverses over gradient flows of the log-posterior with certainly controllable perturbation. Even when the density is not known, existing solutions still can first learn the kernel density models from the given datasets, then produce new samples using the SGLD over the kernel density derivatives. In this work, instead of exploring new samples from kernel spaces, a novel SGLD sampler, namely, Randomized Measurement Langevin Dynamics (RMLD) is proposed to sample the high-dimensional sparse representations from the spectral domain of a given dataset. Specifically, given a random measurement matrix for sparse coding, RMLD first derives a novel likelihood evaluator of the probability distribution from the loss function of LASSO, then samples from the high-dimensional distribution using stochastic Langevin dynamics with derivatives of the logarithm likelihood and Metropolis–Hastings sampling. In addition, new samples in low-dimensional measuring spaces can be regenerated using the sampled high-dimensional vectors and the measurement matrix. The algorithm analysis shows that RMLD indeed projects a given dataset into a high-dimensional Gaussian distribution with Laplacian prior, then draw new sparse representation from the dataset through performing SGLD over the distribution. Extensive experiments have been conducted to evaluate the proposed algorithm using real-world datasets. The performance comparisons on three real-world applications demonstrate the superior performance of RMLD beyond baseline methods.

Aggregated Gradient Langevin Dynamics

Gradient-adjusted underdamped Langevin dynamics for sampling

Utilising the CLT Structure in Stochastic Gradient based Sampling : Improved Analysis and Faster Algorithms

Markov chain Monte Carlo Methods For Lattice Gaussian Sampling:Convergence Analysis and Enhancement

Sampling Sparse Representations with Randomized Measurement Langevin Dynamics

Constrained Ensemble Langevin Monte Carlo

De-randomizing MCMC dynamics with the diffusion Stein operator

Generalized EXTRA stochastic gradient Langevin dynamics

Accelerated Markov Chain Monte Carlo Using Adaptive Weighting Scheme

CPSG-MCMC: Clustering-Based Preprocessing Method for Stochastic Gradient MCMC.

A Langevin-like Sampler for Discrete Distributions

Adaptive Stochastic Gradient Langevin Dynamics: Taming Convergence and Saddle Point Escape Time

Federated Stochastic Gradient Langevin Dynamics

Sequential Controlled Langevin Diffusions

Exact Langevin Dynamics with Stochastic Gradients

Provable Benefit of Annealed Langevin Monte Carlo for Non-log-concave Sampling

Reinforcement Learning for Adaptive MCMC

Langevin Dynamics: A Unified Perspective on Optimization via Lyapunov Potentials

Accelerating Convergence of Replica Exchange Stochastic Gradient MCMC via Variance Reduction

User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient