Abstract:In recent years, it has become increasingly easy to obtain approximate posterior samples via efficient computation algorithms, such as those in variational Bayes. On the other hand, concerns exist on the accuracy of uncertainty estimates, which make it tempting to consider exploiting the approximate samples in canonical Markov chain Monte Carlo algorithms. A major technical barrier is that the approximate sample, when used as a proposal in Metropolis-Hastings steps, tends to have a low acceptance rate as the dimension increases. In this article, we propose a simple yet general solution named ''graph-accelerated Markov Chain Monte Carlo''. We first build a graph with each node location assigned to an approximate sample, then we run Markov chain Monte Carlo with random walks over the graph. In the first stage, we optimize the choice of graph edges to enforce small differences in posterior density/probability between neighboring nodes, while encouraging edges to correspond to large distances in the parameter space. This optimized graph allows us to accelerate a canonical Markov transition kernel through mixing with a large-jump Metropolis-Hastings step, when collecting Markov chain samples at the second stage. Due to its simplicity, this acceleration can be applied to most of the existing Markov chain Monte Carlo algorithms. We theoretically quantify the rate of acceptance as dimension increases, and show the effects on improved mixing time. We demonstrate our approach through improved mixing performances for challenging sampling problems, such as those involving multiple modes, non-convex density contour, or large-dimension latent variables.

Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator

Efficient stochastic optimisation by unadjusted Langevin Monte Carlo. Application to maximum marginal likelihood and empirical Bayesian estimation

Graph-accelerated Markov Chain Monte Carlo using Approximate Samples

Unbiased Markov chain Monte Carlo methods with couplings

Unbiased Markov chain quasi-Monte Carlo for Gibbs samplers

A non-asymptotic error analysis for parallel Monte Carlo estimation from many short Markov chains

A generalized Multiple-try Metropolis version of the Reversible Jump algorithm

Markov chain Monte Carlo without likelihoods

Efficient Bayesian inference for stochastic volatility models with ensemble MCMC methods

Inferring Volatility Dynamics Using Stock Prices and Variance Swap Rates: an Efficient Bayesian Approach

Unbiased Kinetic Langevin Monte Carlo with Inexact Gradients

An effective introduction to the Markov Chain Monte Carlo method

When are Unbiased Monte Carlo Estimators More Preferable than Biased Ones?

Statistical guarantees for stochastic Metropolis-Hastings

Metropolis-Hastings algorithms with autoregressive proposals, and a few examples

Efficient Integrated Volatility Estimation in the Presence of Infinite Variation Jumps via Debiased Truncated Realized Variations

Stochastic Gradient Markov Chain Monte Carlo

Reducing the Variance of Likelihood Ratio Greeks with Monte Carlo

Robust random walk-like Metropolis-Hastings algorithms for concentrating posteriors

Accelerating delayed-acceptance Markov chain Monte Carlo algorithms

Efficient Likelihood-based Estimation Via Annealing for Dynamic Structural Macrofinance Models