Abstract:In Bayesian statistics, exploring multimodal posterior distribution poses major challenges for existing techniques such as Markov Chain Monte Carlo (MCMC). These problems are exacerbated in high-dimensional settings where MCMC methods typically rely upon localised proposal mechanisms. This paper introduces the Annealed Leap-Point Sampler (ALPS), which augments the target distribution state space with modified annealed (cooled) target distributions, in contrast to traditional approaches which have employed tempering. The temperature of the coldest state is chosen such that its corresponding annealed target density can be sufficiently well-approximated by a Laplace approximation. As a result, a Gaussian mixture independence Metropolis-Hastings sampler can perform mode-jumping proposals even in high-dimensional problems. The ability of this procedure to "mode hop" at this super-cold state is then filtered through to the target state using a sequence of tempered targets in a similar way to that in parallel tempering methods. ALPS also incorporates the best aspects of current gold-standard approaches to multimodal sampling in high-dimensional contexts. A theoretical analysis of the ALPS approach in high dimensions is given, providing practitioners with a gauge on the optimal setup as well as the scalability of the algorithm. For a $d$-dimensional problem the it is shown that the coldest inverse temperature level required for the ALPS only needs to be linear in the dimension, $\mathcal{O}(d)$, and this means that for a collection of multimodal problems the algorithmic cost is polynomial, $\mathcal{O}\left(d^{3}\right)$. ALPS is illustrated on a complex multimodal posterior distribution that arises from a seemingly-unrelated regression (SUR) model of longitudinal data from U.S. manufacturing firms.

MALA-within-Gibbs samplers for high-dimensional distributions with sparse conditional structure

On the Computational Complexity of Metropolis-Adjusted Langevin Algorithms for Bayesian Posterior Sampling

A Vine-Copula Based Adaptive Mcmc Sampler for Efficient Inference of Dynamical Systems

Auxiliary MCMC and particle Gibbs samplers for parallelisable inference in latent dynamical systems

The MCMC split sampler: A block Gibbs sampling scheme for latent Gaussian models

Accelerated Markov Chain Monte Carlo Using Adaptive Weighting Scheme

Adapting The Gibbs Sampler

Annealed Leap-Point Sampler for Multimodal Target Distributions

Particle-MALA and Particle-mGRAD: Gradient-based MCMC methods for high-dimensional state-space models

Targeted stochastic gradient Markov chain Monte Carlo for hidden Markov models with rare latent states

Sticky PDMP samplers for sparse and local inference problems

Blocked Gibbs Sampler for Hierarchical Dirichlet Processes

A fast asynchronous MCMC sampler for sparse Bayesian inference

Small World MCMC with Tempering: Ergodicity and Spectral Gap

Fast Conditional Mixing of MCMC Algorithms for Non-log-concave Distributions

Accelerating Multilevel Markov Chain Monte Carlo Using Machine Learning Models

Markov chain Monte Carlo Methods For Lattice Gaussian Sampling:Convergence Analysis and Enhancement

Accelerating delayed-acceptance Markov chain Monte Carlo algorithms

A fresh take on 'Barker dynamics' for MCMC

Infinite dimensional adaptive MCMC for Gaussian processes

Sampling hyperparameters in hierarchical models: improving on Gibbs for high-dimensional latent fields and large data sets