Supplement to "Markov Chain Monte Carlo Based on Deterministic Transformations"

Somak Dutta,Sourabh Bhattacharya

DOI: https://doi.org/10.48550/arXiv.1306.6684

2013-06-28

Computation

Abstract:This is a supplement to the article "Markov Chain Monte Carlo Based on Deterministic Transformations" available at http://arxiv.org/abs/1106.5850

What problem does this paper attempt to address?

A calculus for Markov chain Monte Carlo: studying approximations in algorithms

Rocco Caprio,Adam M. Johansen

2024-09-30

Abstract:Markov chain Monte Carlo (MCMC) algorithms are based on the construction of a Markov chain with transition probabilities leaving invariant a probability distribution of interest. In this work, we look at these transition probabilities as functions of their invariant distributions, and we develop a notion of derivative in the invariant distribution of a MCMC kernel. We build around this concept a set of tools that we refer to as Markov chain Monte Carlo Calculus. This allows us to compare Markov chains with different invariant distributions within a suitable class via what we refer to as mean value inequalities. We explain how MCMC Calculus provides a natural framework to study algorithms using an approximation of an invariant distribution, and we illustrate this by using the tools developed to prove convergence of interacting and sequential MCMC algorithms. Finally, we discuss how similar ideas can be used in other frameworks.

Probability,Statistics Theory,Computation
Markov Chain Monte Carlo (MCMC)

Vivek S. Borkar

DOI: https://doi.org/10.1007/s12045-022-1407-1

2022-07-23

Resonance

Abstract:This is a short account of the major highlights of Markov Chain Monte Carlo, with pointers to what lies beyond.
Geometric ergodicity of trans-dimensional Markov chain Monte Carlo algorithms

Qian Qin

2024-10-17

Abstract:This article studies the convergence properties of trans-dimensional MCMC algorithms when the total number of models is finite. It is shown that, for reversible and some non-reversible trans-dimensional Markov chains, under mild conditions, geometric convergence is guaranteed if the Markov chains associated with the within-model moves are geometrically ergodic. This result is proved in an $L^2$ framework using the technique of Markov chain decomposition. While the technique was previously developed for reversible chains, this work extends it to the point that it can be applied to some commonly used non-reversible chains. The theory herein is applied to reversible jump algorithms for three Bayesian models: a probit regression with variable selection, a Gaussian mixture model with unknown number of components, and an autoregression with Laplace errors and unknown model order.

Statistics Theory
GenMarkov: Modeling Generalized Multivariate Markov Chains in R

Carolina Vasconcelos,Bruno Damásio

DOI: https://doi.org/10.48550/arXiv.2202.00333

2022-02-01

Abstract:This article proposes a new generalization of the Multivariate Markov Chains (MMC) model. The future values of a Markov chain commonly depend on only the past values of the chain in an autoregressive fashion. The generalization proposed in this work also considers exogenous variables that can be deterministic or stochastic. Furthermore, the effects of the MMC's past values and the effects of pre--determined or exogenous covariates are considered in our model by considering a non--homogeneous Markov chain. The Monte Carlo simulation study findings showed that our model consistently detected a non--homogeneous Markov chain. Besides, an empirical illustration demonstrated the relevance of this new model by estimating probability transition matrices over the space state of the exogenous variable. An additional and practical contribution of this work is the development of a novel R package with this generalization.

Methodology
Geometric allocation approaches in Markov chain Monte Carlo

S Todo,H Suwa

DOI: https://doi.org/10.1088/1742-6596/473/1/012013

2013-12-16

Journal of Physics: Conference Series

Abstract:The Markov chain Monte Carlo method is a versatile tool in statistical physics to evaluate multi-dimensional integrals numerically. For the method to work effectively, we must consider the following key issues: the choice of ensemble, the selection of candidate states, the optimization of transition kernel, algorithm for choosing a configuration according to the transition probabilities. We show that the unconventional approaches based on the geometric allocation of probabilities or weights can improve the dynamics and scaling of the Monte Carlo simulation in several aspects. Particularly, the approach using the irreversible kernel can reduce or sometimes completely eliminate the rejection of trial move in the Markov chain. We also discuss how the space-time interchange technique together with Walker's method of aliases can reduce the computational time especially for the case where the number of candidates is large, such as models with long-range interactions.
The importance Markov chain

Charly Andral,Randal Douc,Hugo Marival,Christian P. Robert

DOI: https://doi.org/10.1016/j.spa.2024.104316

IF: 1.43

2024-02-01

Stochastic Processes and their Applications

Abstract:The Importance Markov chain is a novel algorithm bridging the gap between rejection sampling and importance sampling, moving from one to the other through a tuning parameter. Based on a modified sample of an instrumental Markov chain targeting an instrumental distribution (typically via a MCMC kernel), the Importance Markov chain produces an extended Markov chain where the marginal distribution of the first component converges to the target distribution. For example, when targeting a multimodal distribution, the instrumental distribution can be chosen as a tempered version of the target which allows the algorithm to explore its modes more efficiently. We obtain a Law of Large Numbers and a Central Limit Theorem as well as geometric ergodicity for this extended kernel under mild assumptions on the instrumental kernel. Computationally, the algorithm is easy to implement and preexisting librairies can be used to sample from the instrumental distribution.

statistics & probability
Ensemble preconditioning for Markov chain Monte Carlo simulation

Benedict Leimkuhler,Charles Matthews,Jonathan Weare

DOI: https://doi.org/10.1007/s11222-017-9730-1

IF: 2.3241

2017-02-27

Statistics and Computing

Abstract:We describe parallel Markov chain Monte Carlo methods that propagate a collective ensemble of paths, with local covariance information calculated from neighbouring replicas. The use of collective dynamics eliminates multiplicative noise and stabilizes the dynamics, thus providing a practical approach to difficult anisotropic sampling problems in high dimensions. Numerical experiments with model problems demonstrate that dramatic potential speedups, compared to various alternative schemes, are attainable.

statistics & probability,computer science, theory & methods
Parallelization of Markov Chain Generation and its Application to the Multicanonical Method

Takanori Sugihara,Junichi Higo,Haruki Nakamura

DOI: https://doi.org/10.1143/jpsj.78.074003

2009-07-15

Journal of the Physical Society of Japan

Abstract:We develop a simple algorithm to parallelize generation processes of Markov chains. In this algorithm, multiple Markov chains are generated in parallel and jointed together to make a longer Markov chain. The joints between the constituent Markov chains are processed using the detailed balance. We apply the parallelization algorithm to multicanonical calculations of the two-dimensional Ising model and demonstrate accurate estimation of multicanonical weights.

physics, multidisciplinary
Markov chains and mappings of distributions on compact spaces II: Numerics and Conjectures

David J. Aldous,Madelyn Cruz,Shi Feng

2024-03-27

Abstract:Consider a compact metric space $S$ and a pair $(j,k)$ with $k \ge 2$ and $1 \le j \le k$. For any probability distribution $\theta \in P(S)$, define a Markov chain on $S$ by: from state $s$, take $k$ i.i.d. ($\theta$) samples, and jump to the $j$'th closest. Such a chain converges in distribution to a unique stationary distribution, say $\pi_{j,k}(\theta)$. This defines a mapping $\pi_{j,k}: P(S) \to P(S)$. What happens when we iterate this mapping? In particular, what are the fixed points of this mapping? A few results are proved in a companion article; this article, not intended for formal publication, records numerical studies and conjectures.

Probability
Comments on "Particle Markov Chain Monte Carlo" by C. Andrieu, A. Doucet and R. Hollenstein

Julien Cornebise,Gareth W. Peters

DOI: https://doi.org/10.48550/arXiv.0911.3866

2009-11-20

Abstract:We merge in this note our two discussions about the Read Paper "Particle Markov chain Monte Carlo" (Andrieu, Doucet, and Holenstein, 2010) presented on October 16th 2009 at the Royal Statistical Society, appearing in the Journal of the Royal Statistical Society Series B. We also present a more detailed version of the ABC extension.

Methodology,Computation
Markov chains: Structure and applications

R.L. Tweedie

DOI: https://doi.org/10.1016/s0169-7161(01)19025-5

2001-01-01

Handbook of Statistics

Abstract:Publisher The classical theory of Markov chains originally dealt only with chains on finite or countable spaces. In order to establish the fundamental aspects of Markov chain theory on more general spaces, the chapter presents an overview of results on countable spaces, including some recent and less familiar aspects even on these spaces. Although many real systems can be described in such a framework, many other applications require continuous or even more general spaces, and there has been a flourishing of work on non-countable spaces. This chapter integrates the general state space results with topological conditions, which guarantee that the obvious (but not always true) probabilistic properties are preserved when the space has topological features.
Geometry and Dynamics for Markov Chain Monte Carlo

Alessandro Barp,François-Xavier Briol,Anthony D. Kennedy,Mark Girolami

DOI: https://doi.org/10.1146/annurev-statistics-031017-100141

IF: 7.9

2018-03-07

Annual Review of Statistics and Its Application

Abstract:Markov chain Monte Carlo methods have revolutionized mathematical computation and enabled statistical inference within many previously intractable models. In this context, Hamiltonian dynamics have been proposed as an efficient way of building chains that can explore probability densities efficiently. The method emerges from physics and geometry, and these links have been extensively studied over the past thirty years. The aim of this review is to provide a comprehensive introduction to the geometric tools used in Hamiltonian Monte Carlo at a level accessible to statisticians, machine learners, and other users of the methodology with only a basic understanding of Monte Carlo methods. This will be complemented with some discussion of the most recent advances in the field, which we believe will become increasingly relevant to scientists.

statistics & probability,mathematics, interdisciplinary applications
Markov Chain Monte Carlo for Dummies

Masanori Hanada

DOI: https://doi.org/10.48550/arXiv.1808.08490

2018-09-22

Abstract:This is an introductory article about Markov Chain Monte Carlo (MCMC) simulation for pedestrians. Actual simulation codes are provided, and necessary practical details, which are skipped in most textbooks, are shown. The second half is written for hep-th and hep-lat audience. It explains specific methods needed for simulations with dynamical fermions, especially supersymmetric Yang-Mills. The examples include QCD and matrix integral, in addition to SYM.

High Energy Physics - Theory,Statistical Mechanics,High Energy Physics - Lattice
Quantifying Uncertainty in Transdimensional Markov Chain Monte Carlo Using Discrete Markov Models

Daniel W. Heck,Antony M. Overstall,Quentin F. Gronau,Eric-Jan Wagenmakers

DOI: https://doi.org/10.1007/s11222-018-9828-0

2018-07-31

Abstract:Bayesian analysis often concerns an evaluation of models with different dimensionality as is necessary in, for example, model selection or mixture models. To facilitate this evaluation, transdimensional Markov chain Monte Carlo (MCMC) relies on sampling a discrete indexing variable to estimate the posterior model probabilities. However, little attention has been paid to the precision of these estimates. If only few switches occur between the models in the transdimensional MCMC output, precision may be low and assessment based on the assumption of independent samples misleading. Here, we propose a new method to estimate the precision based on the observed transition matrix of the model-indexing variable. Assuming a first order Markov model, the method samples from the posterior of the stationary distribution. This allows assessment of the uncertainty in the estimated posterior model probabilities, model ranks, and Bayes factors. Moreover, the method provides an estimate for the effective sample size of the MCMC output. In two model-selection examples, we show that the proposed approach provides a good assessment of the uncertainty associated with the estimated posterior model probabilities.

Methodology,Computation
Reversible jump Markov chain Monte Carlo and multi-model samplers

Yanan Fan,Scott A. Sisson,Laurence Davies

DOI: https://doi.org/10.48550/arXiv.1001.2055

2024-08-29

Abstract:To appear in the second edition of the MCMC handbook, S. P. Brooks, A. Gelman, G. Jones and X.-L. Meng (eds), Chapman & Hall.

Methodology
Markov Chains Approximations of jump-Diffusion Quantum Trajectories

Clement Pellegrini

DOI: https://doi.org/10.48550/arXiv.0803.2593

2008-03-18

Abstract:"Quantum trajectories" are solutions of stochastic differential equations also called Belavkin or Stochastic Schrödinger Equations. They describe random phenomena in quantum measurement theory. Two types of such equations are usually considered, one is driven by a one-dimensional Brownian motion and the other is driven by a counting process. In this article, we present a way to obtain more advanced models which use jump-diffusion stochastic differential equations. Such models come from solutions of martingale problems for infinitesimal generators. These generators are obtained from the limit of generators of classical Markov chains which describe discrete models of quantum trajectories. Furthermore, stochastic models of jump-diffusion equations are physically justified by proving that their solutions can be obtained as the limit of the discrete trajectories.

Probability
An invitation to adaptive Markov chain Monte Carlo convergence theory

Pietari Laitinen,Matti Vihola

2024-08-27

Abstract:Adaptive Markov chain Monte Carlo (MCMC) algorithms, which automatically tune their parameters based on past samples, have proved extremely useful in practice. The self-tuning mechanism makes them `non-Markovian', which means that their validity cannot be ensured by standard Markov chains theory. Several different techniques have been suggested to analyse their theoretical properties, many of which are technically involved. The technical nature of the theory may make the methods unnecessarily unappealing. We discuss one technique -- based on a martingale decomposition -- with uniformly ergodic Markov transitions. We provide an accessible and self-contained treatment in this setting, and give detailed proofs of the results discussed in the paper, which only require basic understanding of martingale theory and general state space Markov chain concepts. We illustrate how our conditions can accomodate different types of adaptation schemes, and can give useful insight to the requirements which ensure their validity.

Probability,Computation
Fluctuations of Interacting Markov Chain Monte Carlo Methods

Bernard Bercu,Pierre Del Moral,Arnaud Doucet

DOI: https://doi.org/10.48550/arXiv.1201.0480

2012-01-02

Abstract:We present a multivariate central limit theorem for a general class of interacting Markov chain Monte Carlo algorithms used to solve nonlinear measure-valued equations. These algorithms generate stochastic processes which belong to the class of nonlinear Markov chains interacting with their empirical occupation measures. We develop an original theoretical analysis based on resolvent operators and semigroup techniques to analyze the fluctuations of their occupation measures around their limiting values.

Probability,Statistics Theory,Computation,Methodology
An introduction to the Markov chain Monte Carlo method

Wenlong Wang

DOI: https://doi.org/10.1119/5.0122488

IF: 0.835

2022-12-02

American Journal of Physics

Abstract:We present an intuitive, conceptual, and semi-rigorous introduction to the Markov Chain Monte Carlo method using a simple model of population dynamics and focusing on a few elementary distributions. We start from two states, then three states, and finally generalize to many states with both discrete and continuous distributions. Despite the mathematical simplicity, our examples include the essential concepts of the Markov Chain Monte Carlo method, including ergodicity, global balance and detailed balance, proposal or selection probability, acceptance probability, the underlying stochastic matrix, and error analysis. Our experience suggests that most senior undergraduate students in physics can follow these materials without much difficulty.
The Convergence of Markov Chain Monte Carlo Methods: From the Metropolis Method to Hamiltonian Monte Carlo

Michael Betancourt

DOI: https://doi.org/10.1002/andp.201700214

2018-03-23

Annalen der Physik

Abstract:Abstract From its inception in the 1950s to the modern frontiers of applied statistics, Markov chain Monte Carlo has been one of the most ubiquitous and successful methods in statistical computing. The development of the method in that time has been fueled by not only increasingly difficult problems but also novel techniques adopted from physics. Here, the history of Markov chain Monte Carlo is reviewed from its inception with the Metropolis method to the contemporary state‐of‐the‐art in Hamiltonian Monte Carlo, focusing on the evolving interplay between the statistical and physical perspectives of the method.

physics, multidisciplinary

Supplement to "Markov Chain Monte Carlo Based on Deterministic Transformations"

A calculus for Markov chain Monte Carlo: studying approximations in algorithms

Markov Chain Monte Carlo (MCMC)

Geometric ergodicity of trans-dimensional Markov chain Monte Carlo algorithms

GenMarkov: Modeling Generalized Multivariate Markov Chains in R

Geometric allocation approaches in Markov chain Monte Carlo

The importance Markov chain

Ensemble preconditioning for Markov chain Monte Carlo simulation

Parallelization of Markov Chain Generation and its Application to the Multicanonical Method

Markov chains and mappings of distributions on compact spaces II: Numerics and Conjectures

Comments on "Particle Markov Chain Monte Carlo" by C. Andrieu, A. Doucet and R. Hollenstein

Markov chains: Structure and applications

Geometry and Dynamics for Markov Chain Monte Carlo

Markov Chain Monte Carlo for Dummies

Quantifying Uncertainty in Transdimensional Markov Chain Monte Carlo Using Discrete Markov Models

Reversible jump Markov chain Monte Carlo and multi-model samplers

Markov Chains Approximations of jump-Diffusion Quantum Trajectories

An invitation to adaptive Markov chain Monte Carlo convergence theory

Fluctuations of Interacting Markov Chain Monte Carlo Methods

An introduction to the Markov chain Monte Carlo method

The Convergence of Markov Chain Monte Carlo Methods: From the Metropolis Method to Hamiltonian Monte Carlo