Perfecting MCMC Sampling: Recipes and Reservations

Radu V. Craiu,Xiao-Li Meng

DOI: https://doi.org/10.48550/arXiv.2401.02518

2024-01-05

Abstract:This review paper is intended for the Handbook of Markov chain Monte Carlo's second edition. The authors will be grateful for any suggestions that could perfect it.

Computation

What problem does this paper attempt to address?

Data Analysis Recipes: Using Markov Chain Monte Carlo

David W. Hogg,Daniel Foreman-Mackey

DOI: https://doi.org/10.3847/1538-4365/aab76e

2018-05-11

The Astrophysical Journal Supplement Series

Abstract:Markov Chain Monte Carlo (MCMC) methods for sampling probability density functions (combined with abundant computational resources) have transformed the sciences, especially in performing probabilistic inferences, or fitting models to data. In this primarily pedagogical contribution, we give a brief overview of the most basic MCMC method and some practical advice for the use of MCMC in real inference problems. We give advice on method choice, tuning for performance, methods for initialization, tests of convergence, troubleshooting, and use of the chain output to produce or report parameter estimates with associated uncertainties. We argue that autocorrelation time is the most important test for convergence, as it directly connects to the uncertainty on the sampling estimate of any quantity of interest. We emphasize that sampling is a method for doing integrals; this guides our thinking about how MCMC output is best used.

astronomy & astrophysics
Reversible jump Markov chain Monte Carlo and multi-model samplers

Yanan Fan,Scott A. Sisson,Laurence Davies

DOI: https://doi.org/10.48550/arXiv.1001.2055

2024-08-29

Abstract:To appear in the second edition of the MCMC handbook, S. P. Brooks, A. Gelman, G. Jones and X.-L. Meng (eds), Chapman & Hall.

Methodology
Markov Chain Monte Carlo in Practice

Galin L. Jones,Qian Qin

DOI: https://doi.org/10.1146/annurev-statistics-040220-090158

IF: 7.9

2022-03-07

Annual Review of Statistics and Its Application

Abstract:Markov chain Monte Carlo (MCMC) is an essential set of tools for estimating features of probability distributions commonly encountered in modern applications. For MCMC simulation to produce reliable outcomes, it needs to generate observations representative of the target distribution, and it must be long enough so that the errors of Monte Carlo estimates are small. We review methods for assessing the reliability of the simulation effort, with an emphasis on those most useful in practically relevant settings. Both strengths and weaknesses of these methods are discussed. The methods are illustrated in several examples and in a detailed case study.

statistics & probability,mathematics, interdisciplinary applications
Post-Processing of MCMC

Leah F. South,Marina Riabiz,Onur Teymur,Chris. J. Oates

DOI: https://doi.org/10.48550/arXiv.2103.16048

2021-03-30

Methodology

Abstract:Markov chain Monte Carlo (MCMC) is the engine of modern Bayesian statistics, being used to approximate the posterior and derived quantities of interest. Despite this, the issue of how the output from a Markov chain is post-processed and reported is often overlooked. Convergence diagnostics can be used to control bias via burn-in removal, but these do not account for (common) situations where a limited computational budget engenders a bias-variance trade-off. The aim of this article is to review state-of-the-art techniques for post-processing Markov chain output. Our review covers methods based on discrepancy minimisation, which directly address the bias-variance trade-off, as well as general-purpose control variate methods for approximating expected quantities of interest.
Markov Chain Monte Carlo Methods, a survey with some frequent misunderstandings

Christian P. Robert,Wu Changye

DOI: https://doi.org/10.48550/arXiv.2001.06249

2020-01-17

Abstract:In this chapter, we review some of the most standard MCMC tools used in Bayesian computation, along with vignettes on standard misunderstandings of these approaches taken from Q \&~A's on the forum Cross-validated answered by the first author.

Computation
Markov Chain Monte Carlo (MCMC)

Vivek S. Borkar

DOI: https://doi.org/10.1007/s12045-022-1407-1

2022-07-23

Resonance

Abstract:This is a short account of the major highlights of Markov Chain Monte Carlo, with pointers to what lies beyond.
A simple introduction to Markov Chain Monte–Carlo sampling

Don van Ravenzwaaij,Pete Cassey,Scott D. Brown

DOI: https://doi.org/10.3758/s13423-016-1015-8

2016-03-11

Abstract:Markov Chain Monte–Carlo (MCMC) is an increasingly popular method for obtaining information about distributions, especially for estimating posterior distributions in Bayesian inference. This article provides a very basic introduction to MCMC sampling. It describes what MCMC is, and what it can be used for, with simple illustrative examples. Highlighted are some of the benefits and limitations of MCMC sampling, as well as different approaches to circumventing the limitations most likely to trouble cognitive scientists.

psychology, experimental, mathematical
A Conceptual Introduction to Markov Chain Monte Carlo Methods

Joshua S. Speagle

DOI: https://doi.org/10.48550/arXiv.1909.12313

2020-03-08

Abstract:Markov Chain Monte Carlo (MCMC) methods have become a cornerstone of many modern scientific analyses by providing a straightforward approach to numerically estimate uncertainties in the parameters of a model using a sequence of random samples. This article provides a basic introduction to MCMC methods by establishing a strong conceptual understanding of what problems MCMC methods are trying to solve, why we want to use them, and how they work in theory and in practice. To develop these concepts, I outline the foundations of Bayesian inference, discuss how posterior distributions are used in practice, explore basic approaches to estimate posterior-based quantities, and derive their link to Monte Carlo sampling and MCMC. Using a simple toy problem, I then demonstrate how these concepts can be used to understand the benefits and drawbacks of various MCMC approaches. Exercises designed to highlight various concepts are also included throughout the article.

Other Statistics,Instrumentation and Methods for Astrophysics,Data Analysis, Statistics and Probability
Dynamic MCMC Sampling

Weiming Feng,Kun He,Xiaoming Sun,Yitong Yin

2019-01-01

Abstract:The Markov chain Monte Carlo (MCMC) methods are the primary tools for sampling from graphical models, e.g. Markov random fields (MRF). Traditional MCMC sampling algorithms are focused on a classic static setting, where the input is fixed. In this paper we study the problem of sampling from an MRF when the graphical model itself is changing dynamically with time. The problem is well motivated by the growing volume and velocity of data in today's applications of the MCMC methods. For the two major MCMC approaches, respectively for the approximate and perfect sampling, namely, the Gibbs sampling and the coupling from the past (CFTP), we give dynamic versions for the respective MCMC sampling algorithms. On MRF with $n$ variables and bounded maximum degrees, these dynamic sampling algorithms can maintain approximate or perfect samples, while the MRF is dynamically changing. Furthermore, our algorithms are efficient with $\tilde{O}(n)$ space cost, and $\tilde{O}(\log^2n)$ incremental time cost upon each local update to the input MRF, as long as certain decay conditions are satisfied in each step by natural couplings of the corresponding single-site chains. These decay conditions were well known in the literature of couplings for rapid mixing of Markov chains, and now for the first time, are used to imply efficient dynamic sampling algorithms. Consequently, we have efficient dynamic sampling algorithms for the following models: (1) general MRF satisfying the Dobrushin-Shlosman condition (for approximate sampling); (2) Ising model with temperature $\beta$ where $\mathrm{e}^{-2|\beta|}> 1-\frac{2}{\Delta+1}$ (for both approximate and perfect samplings); (3) hardcore model with fugacity $\lambda<\frac{2}{\Delta-2}$ (for both approximate and perfect samplings); (4) proper $q$-coloring with: $q>2\Delta$ (for approximate sampling); or $q>2\Delta^2+3\Delta$ (for perfect sampling).
An invitation to adaptive Markov chain Monte Carlo convergence theory

Pietari Laitinen,Matti Vihola

2024-08-27

Abstract:Adaptive Markov chain Monte Carlo (MCMC) algorithms, which automatically tune their parameters based on past samples, have proved extremely useful in practice. The self-tuning mechanism makes them `non-Markovian', which means that their validity cannot be ensured by standard Markov chains theory. Several different techniques have been suggested to analyse their theoretical properties, many of which are technically involved. The technical nature of the theory may make the methods unnecessarily unappealing. We discuss one technique -- based on a martingale decomposition -- with uniformly ergodic Markov transitions. We provide an accessible and self-contained treatment in this setting, and give detailed proofs of the results discussed in the paper, which only require basic understanding of martingale theory and general state space Markov chain concepts. We illustrate how our conditions can accomodate different types of adaptation schemes, and can give useful insight to the requirements which ensure their validity.

Probability,Computation
Sampling from Dirichlet process mixture models with unknown concentration parameter: Mixing issues in large data implementations

David I. Hastie,Silvia Liverani,Sylvia Richardson

DOI: https://doi.org/10.48550/arXiv.1304.1778

2014-02-21

Abstract:We consider the question of Markov chain Monte Carlo sampling from a general stick-breaking Dirichlet process mixture model, with concentration parameter alpha. This paper introduces a Gibbs sampling algorithm that combines the slice sampling approach of Walker (2007) and the retrospective sampling approach of Papaspiliopoulos and Roberts (2008). Our general algorithm is implemented as efficient open source C++ software, available as an R package, and is based on a blocking strategy similar to that suggested by Papaspiliopoulos (2008) and implemented by Yau et al (2011). We discuss the difficulties of achieving good mixing in MCMC samplers of this nature and investigate sensitivity to initialisation. We additionally consider the challenges when an additional layer of hierarchy is added such that joint inference is to be made on alpha. We introduce a new label switching move and compute the marginal model posterior to help to surmount these difficulties. Our work is illustrated using a profile regression (Molitor et al, 2010) application, where we demonstrate good mixing behaviour for both synthetic and real examples.

Computation
Resampling from the past to improve on MCMC algorithms

Yves F. Atchade

DOI: https://doi.org/10.48550/arXiv.math/0605452

2006-05-17

Abstract:We introduce the idea that resampling from past observations in a Markov Chain Monte Carlo sampler can fasten convergence. We prove that proper resampling from the past does not disturb the limit distribution of the algorithm. We illustrate the method with two examples. The first on a Bayesian analysis of stochastic volatility models and the other on Bayesian phylogeny reconstruction.

Statistics Theory,Probability
Stochastic Gradient Markov Chain Monte Carlo

Christopher Nemeth,Paul Fearnhead

DOI: https://doi.org/10.1080/01621459.2020.1847120

IF: 4.369

2021-01-02

Journal of the American Statistical Association

Abstract:<span><b>Markov chain Monte Carlo (MCMC) algorithms are generally regarded as the gold standard technique for Bayesian inference. They are theoretically well-understood and conceptually simple to apply in practice. The drawback of MCMC is that performing exact inference generally requires all of the data to be processed at each iteration of the algorithm. For large datasets, the computational cost of MCMC can be prohibitive, which has led to recent developments in scalable Monte Carlo algorithms that have a significantly lower computational cost than standard MCMC. In this article, we focus on a particular class of scalable Monte Carlo algorithms, stochastic gradient Markov chain Monte Carlo (SGMCMC) which utilizes data subsampling techniques to reduce the per-iteration cost of MCMC. We provide an introduction to some popular SGMCMC algorithms and review the supporting theoretical results, as well as comparing the efficiency of SGMCMC algorithms against MCMC on benchmark examples. The supporting R code is available online at</b> <a class="ext-link" href="https://github.com/chris-nemeth/sgmcmc-review-paper">https://github.com/chris-nemeth/sgmcmc-review-paper</a>.</span>

statistics & probability
Finding our Way in the Dark: Approximate MCMC for Approximate Bayesian Methods

Evgeny Levi,Radu V. Craiu

DOI: https://doi.org/10.48550/arXiv.1905.06680

2019-05-16

Abstract:With larger data at their disposal, scientists are emboldened to tackle complex questions that require sophisticated statistical models. It is not unusual for the latter to have likelihood functions that elude analytical formulations. Even under such adversity, when one can simulate from the sampling distribution, Bayesian analysis can be conducted using approximate methods such as Approximate Bayesian Computation (ABC) or Bayesian Synthetic Likelihood (BSL). A significant drawback of these methods is that the number of required simulations can be prohibitively large, thus severely limiting their scope. In this paper we design perturbed MCMC samplers that can be used within the ABC and BSL paradigms to significantly accelerate computation while maintaining control on computational efficiency. The proposed strategy relies on recycling samples from the chain's past. The algorithmic design is supported by a theoretical analysis while practical performance is examined via a series of simulation examples and data analyses.

Computation
No Free Lunch for Approximate MCMC

James E. Johndrow,Natesh S. Pillai,Aaron Smith

2024-11-28

Abstract:It is widely known that the performance of Markov chain Monte Carlo (MCMC) can degrade quickly when targeting computationally expensive posterior distributions, such as when the sample size is large. This has motivated the search for MCMC variants that scale well to large datasets. One popular general approach has been to look at only a subsample of the data at every step. In this note, we point out that well-known MCMC convergence results often imply that these ``subsampling'' MCMC algorithms cannot greatly improve performance. We apply these abstract results to realistic statistical problems and proposed algorithms, and also discuss some design principles suggested by the results. Finally, we develop estimates for the singular values of random matrices bounds that may be of independent interest.

Computation,Statistics Theory
Markov Chain Sampling Methods for Dirichlet Process Mixture Models

Radford M. Neal

DOI: https://doi.org/10.1080/10618600.2000.10474879

2000-06-01

Journal of Computational and Graphical Statistics

Abstract:This article reviews Markov chain methods for sampling from the posterior distribution of a Dirichlet process mixture model and presents two new classes of methods. One new approach is to make Metropolis—Hastings updates of the indicators specifying which mixture component is associated with each observation, perhaps supplemented with a partial form of Gibbs sampling. The other new approach extends Gibbs sampling for these indicators by using a set of auxiliary parameters. These methods are simple to implement and are more efficient than previous ways of handling general Dirichlet process mixture models with non-conjugate priors.

statistics & probability
Efficiency and Parameter Selection of a micro-macro Markov chain Monte Carlo method for molecular dynamics

Hannes Vandecasteele,Giovanni Samaey

DOI: https://doi.org/10.48550/arXiv.2209.13056

2022-09-27

Abstract:We recently introduced a mM-MCMC scheme that is able to accelerate the sampling of Gibbs distributions when there is a time-scale separation between the complete molecular dynamics and the slow dynamics of a low dimensional reaction coordinate. The mM-MCMC Markov chain works in three steps: 1) compute the reaction coordinate value associated to the current molecular state; 2) generate a new macroscopic proposal using some approximate macroscopic distribution; 3) reconstruct a molecular configuration that is consistent with the newly sampled macroscopic value. There are a number of method parameters that impact the efficiency of the mM-MCMC method. On the macroscopic level, the proposal- and approximate macroscopic distributions are important, while on the microscopic level the reconstruction distribution is of significant importance. In this manuscript, we will investigate the impact of these parameters on the efficiency of the mM-MCMC method on two molecules: a simple three-atom molecule and butane.

Numerical Analysis
Convergence Speed and Approximation Accuracy of Numerical MCMC

Tiangang Cui,Jing Dong,Ajay Jasra,Xin T. Tong

DOI: https://doi.org/10.48550/arXiv.2203.03104

2022-03-07

Abstract:When implementing Markov Chain Monte Carlo (MCMC) algorithms, perturbation caused by numerical errors is sometimes inevitable. This paper studies how perturbation of MCMC affects the convergence speed and Monte Carlo estimation accuracy. Our results show that when the original Markov chain converges to stationarity fast enough and the perturbed transition kernel is a good approximation to the original transition kernel, the corresponding perturbed sampler has similar convergence speed and high approximation accuracy as well. We discuss two different analysis frameworks: ergodicity and spectral gap, both are widely used in the literature. Our results can be easily extended to obtain non-asymptotic error bounds for MCMC estimators. We also demonstrate how to apply our convergence and approximation results to the analysis of specific sampling algorithms, including Random walk Metropolis and Metropolis adjusted Langevin algorithm with perturbed target densities, and parallel tempering Monte Carlo with perturbed densities. Finally we present some simple numerical examples to verify our theoretical claims.

Computation,Probability
Markov chain Monte Carlo methods in biostatistics

Andrew Gelman,Donald B Rubin

DOI: https://doi.org/10.1177/096228029600500402

IF: 2.494

1996-12-01

Statistical Methods in Medical Research

Abstract:Appropriate models in biostatistics are often quite complicated. Such models are typically most easily fit using Bayesian methods, which can often be implemented using simulation techniques. Markov chain Monte Carlo (MCMC) methods are an important set of tools for such simulations. We give an overview and references of this rapidly emerging technology along with a relatively simple example. MCMC techniques can be viewed as extensions of iterative maximization techniques, but with random jumps rather than maximizations at each step. Special care is needed when implementing iterative maximization procedures rather than closed-form methods, and even more care is needed with iterative simulation procedures: it is substantially more difficult to monitor convergence to a distribution than to a point. The most reliable implementations of MCMC build upon results from simpler models fit using combinations of maximization algorithms and noniterative simulations, so that the user has a rough idea of the location and scale of the posterior distribution of the quantities of interest under the more complicated model. These concerns with implementation, however, should not deter the biostatistician from using MCMC methods, but rather help to ensure wise use of these powerful techniques.

health care sciences & services,medical informatics,mathematical & computational biology,statistics & probability
Multicanonical MCMC for Sampling Rare Events

Yukito Iba,Nen Saito,Akimasa Kitajima

DOI: https://doi.org/10.1007/s10463-014-0460-2

2014-10-17

Abstract:Multicanonical MCMC (Multicanonical Markov Chain Monte Carlo; Multicanonical Monte Carlo) is discussed as a method of rare event sampling. Starting from a review of the generic framework of importance sampling, multicanonical MCMC is introduced, followed by applications in random matrices, random graphs, and chaotic dynamical systems. Replica exchange MCMC (also known as parallel tempering or Metropolis-coupled MCMC) is also explained as an alternative to multicanonical MCMC. In the last section, multicanonical MCMC is applied to data surrogation; a successful implementation in surrogating time series is shown. In the appendices, calculation of averages and normalizing constant in an exponential family, phase coexistence, simulated tempering, parallelization, and multivariate extensions are discussed.

Statistical Mechanics,Disordered Systems and Neural Networks,Chaotic Dynamics,Computational Physics,Data Analysis, Statistics and Probability

Perfecting MCMC Sampling: Recipes and Reservations

Data Analysis Recipes: Using Markov Chain Monte Carlo

Reversible jump Markov chain Monte Carlo and multi-model samplers

Markov Chain Monte Carlo in Practice

Post-Processing of MCMC

Markov Chain Monte Carlo Methods, a survey with some frequent misunderstandings

Markov Chain Monte Carlo (MCMC)

A simple introduction to Markov Chain Monte–Carlo sampling

A Conceptual Introduction to Markov Chain Monte Carlo Methods

Dynamic MCMC Sampling

An invitation to adaptive Markov chain Monte Carlo convergence theory

Sampling from Dirichlet process mixture models with unknown concentration parameter: Mixing issues in large data implementations

Resampling from the past to improve on MCMC algorithms

Stochastic Gradient Markov Chain Monte Carlo

Finding our Way in the Dark: Approximate MCMC for Approximate Bayesian Methods

No Free Lunch for Approximate MCMC

Markov Chain Sampling Methods for Dirichlet Process Mixture Models

Efficiency and Parameter Selection of a micro-macro Markov chain Monte Carlo method for molecular dynamics

Convergence Speed and Approximation Accuracy of Numerical MCMC

Markov chain Monte Carlo methods in biostatistics

Multicanonical MCMC for Sampling Rare Events