Reversible jump Markov chain Monte Carlo and multi-model samplers

Yanan Fan,Scott A. Sisson,Laurence Davies

DOI: https://doi.org/10.48550/arXiv.1001.2055

2024-08-29

Abstract:To appear in the second edition of the MCMC handbook, S. P. Brooks, A. Gelman, G. Jones and X.-L. Meng (eds), Chapman & Hall.

Methodology

What problem does this paper attempt to address?

Markov Chain Monte Carlo (MCMC)

Vivek S. Borkar

DOI: https://doi.org/10.1007/s12045-022-1407-1

2022-07-23

Resonance

Abstract:This is a short account of the major highlights of Markov Chain Monte Carlo, with pointers to what lies beyond.
MCMC-driven learning

Alexandre Bouchard-Côté,Trevor Campbell,Geoff Pleiss,Nikola Surjanovic

2024-02-15

Abstract:This paper is intended to appear as a chapter for the Handbook of Markov Chain Monte Carlo. The goal of this chapter is to unify various problems at the intersection of Markov chain Monte Carlo (MCMC) and machine learning$\unicode{x2014}$which includes black-box variational inference, adaptive MCMC, normalizing flow construction and transport-assisted MCMC, surrogate-likelihood MCMC, coreset construction for MCMC with big data, Markov chain gradient descent, Markovian score climbing, and more$\unicode{x2014}$within one common framework. By doing so, the theory and methods developed for each may be translated and generalized.

Machine Learning,Statistics Theory,Computation
Markov Chain Monte Carlo for Continuous-Time Switching Dynamical Systems

Lukas Köhs,Bastian Alt,Heinz Koeppl

DOI: https://doi.org/10.48550/arXiv.2205.08803

2022-05-18

Abstract:Switching dynamical systems are an expressive model class for the analysis of time-series data. As in many fields within the natural and engineering sciences, the systems under study typically evolve continuously in time, it is natural to consider continuous-time model formulations consisting of switching stochastic differential equations governed by an underlying Markov jump process. Inference in these types of models is however notoriously difficult, and tractable computational schemes are rare. In this work, we propose a novel inference algorithm utilizing a Markov Chain Monte Carlo approach. The presented Gibbs sampler allows to efficiently obtain samples from the exact continuous-time posterior processes. Our framework naturally enables Bayesian parameter estimation, and we also include an estimate for the diffusion covariance, which is oftentimes assumed fixed in stochastic differential equation models. We evaluate our framework under the modeling assumption and compare it against an existing variational inference approach.

Machine Learning
Scalable MCMC for Mixed Membership Stochastic Blockmodels

Wenzhe Li,Sungjin Ahn,Max Welling

DOI: https://doi.org/10.48550/arXiv.1510.04815

IF: 5.414

2015-10-16

Machine Learning

Abstract:We propose a stochastic gradient Markov chain Monte Carlo (SG-MCMC) algorithm for scalable inference in mixed-membership stochastic blockmodels (MMSB). Our algorithm is based on the stochastic gradient Riemannian Langevin sampler and achieves both faster speed and higher accuracy at every iteration than the current state-of-the-art algorithm based on stochastic variational inference. In addition we develop an approximation that can handle models that entertain a very large number of communities. The experimental results show that SG-MCMC strictly dominates competing algorithms in all cases.
Perfecting MCMC Sampling: Recipes and Reservations

Radu V. Craiu,Xiao-Li Meng

DOI: https://doi.org/10.48550/arXiv.2401.02518

2024-01-05

Abstract:This review paper is intended for the Handbook of Markov chain Monte Carlo's second edition. The authors will be grateful for any suggestions that could perfect it.

Computation
Markov Chain Monte Carlo in Practice

Galin L. Jones,Qian Qin

DOI: https://doi.org/10.1146/annurev-statistics-040220-090158

IF: 7.9

2022-03-07

Annual Review of Statistics and Its Application

Abstract:Markov chain Monte Carlo (MCMC) is an essential set of tools for estimating features of probability distributions commonly encountered in modern applications. For MCMC simulation to produce reliable outcomes, it needs to generate observations representative of the target distribution, and it must be long enough so that the errors of Monte Carlo estimates are small. We review methods for assessing the reliability of the simulation effort, with an emphasis on those most useful in practically relevant settings. Both strengths and weaknesses of these methods are discussed. The methods are illustrated in several examples and in a detailed case study.

statistics & probability,mathematics, interdisciplinary applications
Markov Chain Monte Carlo and Variational Inference: Bridging the Gap

Tim Salimans,Diederik P. Kingma,Max Welling

DOI: https://doi.org/10.48550/arXiv.1410.6460

2015-05-19

Abstract:Recent advances in stochastic gradient variational inference have made it possible to perform variational Bayesian inference with posterior approximations containing auxiliary random variables. This enables us to explore a new synthesis of variational inference and Monte Carlo methods where we incorporate one or more steps of MCMC into our variational approximation. By doing so we obtain a rich class of inference algorithms bridging the gap between variational methods and MCMC, and offering the best of both worlds: fast posterior approximation through the maximization of an explicit objective, with the option of trading off additional computation for additional accuracy. We describe the theoretical foundations that make this possible and show some promising first results.

Computation,Machine Learning
Markov Chain Monte Carlo for Dummies

Masanori Hanada

DOI: https://doi.org/10.48550/arXiv.1808.08490

2018-09-22

Abstract:This is an introductory article about Markov Chain Monte Carlo (MCMC) simulation for pedestrians. Actual simulation codes are provided, and necessary practical details, which are skipped in most textbooks, are shown. The second half is written for hep-th and hep-lat audience. It explains specific methods needed for simulations with dynamical fermions, especially supersymmetric Yang-Mills. The examples include QCD and matrix integral, in addition to SYM.

High Energy Physics - Theory,Statistical Mechanics,High Energy Physics - Lattice
The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method

Alexandre Bouchard-Côté,Sebastian J. Vollmer,Arnaud Doucet

DOI: https://doi.org/10.1080/01621459.2017.1294075

IF: 4.369

2018-04-03

Journal of the American Statistical Association

Abstract:Many Markov chain Monte Carlo techniques currently available rely on discrete-time reversible Markov processes whose transition kernels are variations of the Metropolis–Hastings algorithm. We explore and generalize an alternative scheme recently introduced in the physics literature (Peters and de With 2012) where the target distribution is explored using a continuous-time nonreversible piecewise-deterministic Markov process. In the Metropolis–Hastings algorithm, a trial move to a region of lower target density, equivalently of higher “energy,” than the current state can be rejected with positive probability. In this alternative approach, a particle moves along straight lines around the space and, when facing a high energy barrier, it is not rejected but its path is modified by bouncing against this barrier. By reformulating this algorithm using inhomogeneous Poisson processes, we exploit standard sampling techniques to simulate exactly this Markov process in a wide range of scenarios of interest. Additionally, when the target distribution is given by a product of factors dependent only on subsets of the state variables, such as the posterior distribution associated with a probabilistic graphical model, this method can be modified to take advantage of this structure by allowing computationally cheaper “local” bounces, which only involve the state variables associated with a factor, while the other state variables keep on evolving. In this context, by leveraging techniques from chemical kinetics, we propose several computationally efficient implementations. Experimentally, this new class of Markov chain Monte Carlo schemes compares favorably to state-of-the-art methods on various Bayesian inference tasks, including for high-dimensional models and large datasets. Supplementary materials for this article are available online.

statistics & probability
Fast MCMC sampling for Markov jump processes and continuous time Bayesian networks

Vinayak Rao,Yee Whye Teh

DOI: https://doi.org/10.48550/arXiv.1202.3760

2012-02-14

Methodology

Abstract:Markov jump processes and continuous time Bayesian networks are important classes of continuous time dynamical systems. In this paper, we tackle the problem of inferring unobserved paths in these models by introducing a fast auxiliary variable Gibbs sampler. Our approach is based on the idea of uniformization, and sets up a Markov chain over paths by sampling a finite set of virtual jump times and then running a standard hidden Markov model forward filtering-backward sampling algorithm over states at the set of extant and virtual jump times. We demonstrate significant computational benefits over a state-of-the-art Gibbs sampler on a number of continuous time Bayesian networks.
Irreversible Samplers from Jump and Continuous Markov Processes.

Yi-An Ma,Emily B. Fox,Tianqi Chen,Lei Wu

DOI: https://doi.org/10.1007/s11222-018-9802-x

IF: 2.3241

2018-01-01

Statistics and Computing

Abstract:In this paper, we propose irreversible versions of the Metropolis Hastings (MH) and Metropolis adjusted Langevin algorithm (MALA) with a main focus on the latter. For the former, we show how one can simply switch between different proposal and acceptance distributions upon rejection to obtain an irreversible jump sampler (I-Jump). The resulting algorithm has a simple implementation akin to MH, but with the demonstrated benefits of irreversibility. We then show how the previously proposed MALA method can also be extended to exploit irreversible stochastic dynamics as proposal distributions in the I-Jump sampler. Our experiments explore how irreversibility can increase the efficiency of the samplers in different situations.
Improved MCMC Method and Its Application

刘国华,朱嵩,黄跃飞,毛根海

DOI: https://doi.org/10.3321/j.issn:0559-9350.2009.08.018

2009-01-01

Abstract:A multi-chain sampling method based on Metropolis-Hastings algorithm was used to improve the Markov Chain Monte Carlo(MCMC) method in order to prevent from trapped into the local optimal solutions that often occur to probability inversion by using current MCMC algorithm.The improved MCMC method can adjust the number of chains according to the feedback results from sampling process in real time,so that it can search out the non-unique solutions as much as possible while saving the time of multi-chain search.As an example an inverse problem of groundwater flow was solved by using the improved MCMC algorithm.The computational results indicate the improved method performs well in solving inverse problems with non-unique solutions.
Accelerating delayed-acceptance Markov chain Monte Carlo algorithms

Samuel Wiqvist,Umberto Picchini,Julie Lyng Forman,Kresten Lindorff-Larsen,Wouter Boomsma

DOI: https://doi.org/10.48550/arXiv.1806.05982

2018-06-15

Computation

Abstract:Delayed-acceptance Markov chain Monte Carlo (DA-MCMC) samples from a probability distribution via a two-stages version of the Metropolis-Hastings algorithm, by combining the target distribution with a "surrogate" (i.e. an approximate and computationally cheaper version) of said distribution. DA-MCMC accelerates MCMC sampling in complex applications, while still targeting the exact distribution. We design a computationally faster, albeit approximate, DA-MCMC algorithm. We consider parameter inference in a Bayesian setting where a surrogate likelihood function is introduced in the delayed-acceptance scheme. When the evaluation of the likelihood function is computationally intensive, our scheme produces a 2-4 times speed-up, compared to standard DA-MCMC. However, the acceleration is highly problem dependent. Inference results for the standard delayed-acceptance algorithm and our approximated version are similar, indicating that our algorithm can return reliable Bayesian inference. As a computationally intensive case study, we introduce a novel stochastic differential equation model for protein folding data.
Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems

Benjamin Ballnus,Sabine Hug,Kathrin Hatz,Linus Görlitz,Jan Hasenauer,Fabian J Theis

DOI: https://doi.org/10.1186/s12918-017-0433-1

2017-06-24

Abstract:Background: In quantitative biology, mathematical models are used to describe and analyze biological processes. The parameters of these models are usually unknown and need to be estimated from experimental data using statistical methods. In particular, Markov chain Monte Carlo (MCMC) methods have become increasingly popular as they allow for a rigorous analysis of parameter and prediction uncertainties without the need for assuming parameter identifiability or removing non-identifiable parameters. A broad spectrum of MCMC algorithms have been proposed, including single- and multi-chain approaches. However, selecting and tuning sampling algorithms suited for a given problem remains challenging and a comprehensive comparison of different methods is so far not available. Results: We present the results of a thorough benchmarking of state-of-the-art single- and multi-chain sampling methods, including Adaptive Metropolis, Delayed Rejection Adaptive Metropolis, Metropolis adjusted Langevin algorithm, Parallel Tempering and Parallel Hierarchical Sampling. Different initialization and adaptation schemes are considered. To ensure a comprehensive and fair comparison, we consider problems with a range of features such as bifurcations, periodical orbits, multistability of steady-state solutions and chaotic regimes. These problem properties give rise to various posterior distributions including uni- and multi-modal distributions and non-normally distributed mode tails. For an objective comparison, we developed a pipeline for the semi-automatic comparison of sampling results. Conclusion: The comparison of MCMC algorithms, initialization and adaptation schemes revealed that overall multi-chain algorithms perform better than single-chain algorithms. In some cases this performance can be further increased by using a preceding multi-start local optimization scheme. These results can inform the selection of sampling methods and the benchmark collection can serve for the evaluation of new algorithms. Furthermore, our results confirm the need to address exploration quality of MCMC chains before applying the commonly used quality measure of effective sample size to prevent false analysis conclusions.
An invitation to adaptive Markov chain Monte Carlo convergence theory

Pietari Laitinen,Matti Vihola

2024-08-27

Abstract:Adaptive Markov chain Monte Carlo (MCMC) algorithms, which automatically tune their parameters based on past samples, have proved extremely useful in practice. The self-tuning mechanism makes them `non-Markovian', which means that their validity cannot be ensured by standard Markov chains theory. Several different techniques have been suggested to analyse their theoretical properties, many of which are technically involved. The technical nature of the theory may make the methods unnecessarily unappealing. We discuss one technique -- based on a martingale decomposition -- with uniformly ergodic Markov transitions. We provide an accessible and self-contained treatment in this setting, and give detailed proofs of the results discussed in the paper, which only require basic understanding of martingale theory and general state space Markov chain concepts. We illustrate how our conditions can accomodate different types of adaptation schemes, and can give useful insight to the requirements which ensure their validity.

Probability,Computation
A generalized Multiple-try Metropolis version of the Reversible Jump algorithm

S. Pandolfi,F. Bartolucci,N. Friel

DOI: https://doi.org/10.48550/arXiv.1006.0621

2010-06-03

Methodology

Abstract:The Reversible Jump algorithm is one of the most widely used Markov chain Monte Carlo algorithms for Bayesian estimation and model selection. A generalized multiple-try version of this algorithm is proposed. The algorithm is based on drawing several proposals at each step and randomly choosing one of them on the basis of weights (selection probabilities) that may be arbitrary chosen. Among the possible choices, a method is employed which is based on selection probabilities depending on a quadratic approximation of the posterior distribution. Moreover, the implementation of the proposed algorithm for challenging model selection problems, in which the quadratic approximation is not feasible, is considered. The resulting algorithm leads to a gain in efficiency with respect to the Reversible Jump algorithm, and also in terms of computational effort. The performance of this approach is illustrated for real examples involving a logistic regression model and a latent class model.
Unbiased Markov chain Monte Carlo methods with couplings

Pierre E. Jacob,John O’Leary,Yves F. Atchadé

DOI: https://doi.org/10.1111/rssb.12336

2020-05-06

Abstract:Markov chain Monte Carlo (MCMC) methods provide consistent approximations of integrals as the number of iterations goes to ∞. MCMC estimators are generally biased after any fixed number of iterations. We propose to remove this bias by using couplings of Markov chains together with a telescopic sum argument of Glynn and Rhee. The resulting unbiased estimators can be computed independently in parallel. We discuss practical couplings for popular MCMC algorithms. We establish the theoretical validity of the estimators proposed and study their efficiency relative to the underlying MCMC algorithms. Finally, we illustrate the performance and limitations of the method on toy examples, on an Ising model around its critical temperature, on a high dimensional variable‐selection problem, and on an approximation of the cut distribution arising in Bayesian inference for models made of multiple modules.

statistics & probability
Markov Chain Monte Carlo Methods, a survey with some frequent misunderstandings

Christian P. Robert,Wu Changye

DOI: https://doi.org/10.48550/arXiv.2001.06249

2020-01-17

Abstract:In this chapter, we review some of the most standard MCMC tools used in Bayesian computation, along with vignettes on standard misunderstandings of these approaches taken from Q \&~A's on the forum Cross-validated answered by the first author.

Computation
Dynamic MCMC Sampling

Weiming Feng,Kun He,Xiaoming Sun,Yitong Yin

2019-01-01

Abstract:The Markov chain Monte Carlo (MCMC) methods are the primary tools for sampling from graphical models, e.g. Markov random fields (MRF). Traditional MCMC sampling algorithms are focused on a classic static setting, where the input is fixed. In this paper we study the problem of sampling from an MRF when the graphical model itself is changing dynamically with time. The problem is well motivated by the growing volume and velocity of data in today's applications of the MCMC methods. For the two major MCMC approaches, respectively for the approximate and perfect sampling, namely, the Gibbs sampling and the coupling from the past (CFTP), we give dynamic versions for the respective MCMC sampling algorithms. On MRF with $n$ variables and bounded maximum degrees, these dynamic sampling algorithms can maintain approximate or perfect samples, while the MRF is dynamically changing. Furthermore, our algorithms are efficient with $\tilde{O}(n)$ space cost, and $\tilde{O}(\log^2n)$ incremental time cost upon each local update to the input MRF, as long as certain decay conditions are satisfied in each step by natural couplings of the corresponding single-site chains. These decay conditions were well known in the literature of couplings for rapid mixing of Markov chains, and now for the first time, are used to imply efficient dynamic sampling algorithms. Consequently, we have efficient dynamic sampling algorithms for the following models: (1) general MRF satisfying the Dobrushin-Shlosman condition (for approximate sampling); (2) Ising model with temperature $\beta$ where $\mathrm{e}^{-2|\beta|}> 1-\frac{2}{\Delta+1}$ (for both approximate and perfect samplings); (3) hardcore model with fugacity $\lambda<\frac{2}{\Delta-2}$ (for both approximate and perfect samplings); (4) proper $q$-coloring with: $q>2\Delta$ (for approximate sampling); or $q>2\Delta^2+3\Delta$ (for perfect sampling).
Efficient Irreversible Monte Carlo Samplers

Fahim Faizi,George Deligiannidis,Edina Rosta

DOI: https://doi.org/10.1021/acs.jctc.9b01135

2020-02-25

Journal of Chemical Theory and Computation

Abstract:We present here two irreversible Markov chain Monte Carlo algorithms for general discrete state systems. One of the algorithms is based on the random-scan Gibbs sampler for discrete states and the other on its improved version, the Metropolized-Gibbs sampler. The algorithms we present incorporate the lifting framework with skewed detailed balance condition and construct irreversible Markov chains that satisfy the balance condition. We have applied our algorithms to 1D 4-state Potts model. The integrated autocorrelation times for magnetization and energy density indicate a reduction of the dynamical scaling exponent from z ≈ 1 to z ≈ 1/2. In addition, we have generalized an irreversible Metropolis–Hastings algorithm with skewed detailed balance, initially introduced by Turitsyn et al. [ Physica D 2011, 240, 410] for the mean field Ising model, to be now readily applicable to classical spin systems in general; application to 1D 4-state Potts model indicate a square root reduction of the mixing time at high temperatures.

chemistry, physical,physics, atomic, molecular & chemical

Reversible jump Markov chain Monte Carlo and multi-model samplers

Markov Chain Monte Carlo (MCMC)

MCMC-driven learning

Markov Chain Monte Carlo for Continuous-Time Switching Dynamical Systems

Scalable MCMC for Mixed Membership Stochastic Blockmodels

Perfecting MCMC Sampling: Recipes and Reservations

Markov Chain Monte Carlo in Practice

Markov Chain Monte Carlo and Variational Inference: Bridging the Gap

Markov Chain Monte Carlo for Dummies

The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method

Fast MCMC sampling for Markov jump processes and continuous time Bayesian networks

Irreversible Samplers from Jump and Continuous Markov Processes.

Improved MCMC Method and Its Application

Accelerating delayed-acceptance Markov chain Monte Carlo algorithms

Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems

An invitation to adaptive Markov chain Monte Carlo convergence theory

A generalized Multiple-try Metropolis version of the Reversible Jump algorithm

Unbiased Markov chain Monte Carlo methods with couplings

Markov Chain Monte Carlo Methods, a survey with some frequent misunderstandings

Dynamic MCMC Sampling

Efficient Irreversible Monte Carlo Samplers