Sequential monte carlo samplers

Pierre Del Moral,Arnaud Doucet

DOI: https://doi.org/10.48550/arXiv.cond-mat/0212648

2002-12-31

Abstract:This paper shows how one can use Sequential Monte Carlo methods to perform what is typically done using Markov chain Monte Carlo methods. This leads to a general class of principled integration and genetic type optimization methods based on interacting particle systems.

Condensed Matter

What problem does this paper attempt to address?

An invitation to sequential Monte Carlo samplers

Chenguang Dai,Jeremy Heng,Pierre E. Jacob,Nick Whiteley

DOI: https://doi.org/10.48550/arXiv.2007.11936

2022-06-17

Abstract:Statisticians often use Monte Carlo methods to approximate probability distributions, primarily with Markov chain Monte Carlo and importance sampling. Sequential Monte Carlo samplers are a class of algorithms that combine both techniques to approximate distributions of interest and their normalizing constants. These samplers originate from particle filtering for state space models and have become general and scalable sampling techniques. This article describes sequential Monte Carlo samplers and their possible implementations, arguing that they remain under-used in statistics, despite their ability to perform sequential inference and to leverage parallel processing resources among other potential benefits.

Computation,Methodology
Sequential Monte Carlo with Highly Informative Observations

Pierre Del Moral,Lawrence M. Murray

DOI: https://doi.org/10.48550/arXiv.1405.4081

2015-07-09

Abstract:We propose sequential Monte Carlo (SMC) methods for sampling the posterior distribution of state-space models under highly informative observation regimes, a situation in which standard SMC methods can perform poorly. A special case is simulating bridges between given initial and final values. The basic idea is to introduce a schedule of intermediate weighting and resampling times between observation times, which guide particles towards the final state. This can always be done for continuous-time models, and may be done for discrete-time models under sparse observation regimes; our main focus is on continuous-time diffusion processes. The methods are broadly applicable in that they support multivariate models with partial observation, do not require simulation of the backward transition (which is often unavailable), and, where possible, avoid pointwise evaluation of the forward transition. When simulating bridges, the last cannot be avoided entirely without concessions, and we suggest an epsilon-ball approach (reminiscent of Approximate Bayesian Computation) as a workaround. Compared to the bootstrap particle filter, the new methods deliver substantially reduced mean squared error in normalising constant estimates, even after accounting for execution time. The methods are demonstrated for state estimation with two toy examples, and for parameter estimation (within a particle marginal Metropolis--Hastings sampler) with three applied examples in econometrics, epidemiology and marine biogeochemistry.

Computation,Applications
An Introduction to Sequential Monte Carlo

Nicolas Chopin,Omiros Papaspiliopoulos

DOI: https://doi.org/10.1007/978-3-030-47845-2

2020-01-01

Abstract:This book provides a general introduction to Sequential Monte Carlo (SMC) methods, also known as particle filters. These methods have become a staple for the sequential analysis of data in such diverse fields as signal processing, epidemiology, machine learning, population ecology, quantitative finance, and robotics. The coverage is comprehensive, ranging from the underlying theory to computational implementation, methodology, and diverse applications in various areas of science. This is achieved by describing SMC algorithms as particular cases of a general framework, which involves concepts such as Feynman-Kac distributions, and tools such as importance sampling and resampling. This general framework is used consistently throughout the book. Extensive coverage is provided on sequential learning (filtering, smoothing) of state-space (hidden Markov) models, as this remains an important application of SMC methods. More recent applications, such as parameter estimation of these models (through e.g. particle Markov chain Monte Carlo techniques) and the simulation of challenging probability distributions (in e.g. Bayesian inference or rare-event problems), are also discussed. The book may be used either as a graduate text on Sequential Monte Carlo methods and state-space modeling, or as a general reference work on the area. Each chapter includes a set of exercises for self-study, a comprehensive bibliography, and a “Python corner,” which discusses the practical implementation of the methods covered. In addition, the book comes with an open source Python library, which implements all the algorithms described in the book, and contains all the programs that were used to perform the numerical experiments.
On sequential Monte Carlo sampling methods for Bayesian filtering

Arnaud Doucet,Simon Godsill,Christophe Andrieu

DOI: https://doi.org/10.1023/a:1008935410038

IF: 2.3241

2000-01-01

Statistics and Computing

Abstract:In this article, we present an overview of methods for sequential simulation from posterior distributions. These methods are of particular interest in Bayesian filtering for discrete time dynamic models that are typically nonlinear and non-Gaussian. A general importance sampling framework is developed that unifies many of the methods which have been proposed over the last few decades in several different scientific disciplines. Novel extensions to the existing methods are also proposed. We show in particular how to incorporate local linearisation methods similar to those which have previously been employed in the deterministic filtering literature; these lead to very effective importance distributions. Furthermore we describe a method which uses Rao-Blackwellisation in order to take advantage of the analytic structure present in some important classes of state-space models. In a final section we develop algorithms for prediction, smoothing and evaluation of the likelihood in dynamic models.

statistics & probability,computer science, theory & methods
Convergence of sequential Markov Chain Monte Carlo methods: I. Nonlinear flow of probability measures

Andreas Eberle,Carlo Marinelli

DOI: https://doi.org/10.48550/arXiv.math/0612074

2006-12-03

Probability

Abstract:Sequential Monte Carlo Samplers are a class of stochastic algorithms for Monte Carlo integral estimation w.r.t. probability distributions, which combine elements of Markov chain Monte Carlo methods and importance sampling/resampling schemes. We develop a stability analysis by functional inequalities for a nonlinear flow of probability measures describing the limit behavior of the algorithms as the number of particles tends to infinity. Stability results are derived both under global and local assumptions on the generator of the underlying Metropolis dynamics. This allows us to prove that the combined methods sometimes have good asymptotic stability properties in multimodal setups where traditional MCMC methods mix extremely slowly. For example, this holds for the mean field Ising model at all temperatures.
Multicanonical sequential Monte Carlo sampler for uncertainty quantification

Robert Millar,Hui Li,Jinglai Li

DOI: https://doi.org/10.1016/j.ress.2023.109316

2023-04-22

Abstract:In many real-world engineering systems, the performance or reliability of the system is characterised by a scalar variable. The distribution of this performance variable is important in many uncertainty quantification problems, ranging from risk management to utility optimisation. In practice, this distribution usually cannot be derived analytically and has to be obtained numerically by simulations. To this end, standard Monte Carlo simulations are often used, however, they cannot efficiently reconstruct the tail of the distribution which is essential in many applications. One possible remedy is to use the Multicanonical Monte Carlo method, an adaptive importance sampling scheme. In this method, one draws samples from an importance sampling distribution in a nonstandard form in each iteration, which is usually done via Markov chain Monte Carlo (MCMC). MCMC is inherently serial and therefore struggles with parallelism. In this paper, we present a new approach, which uses the Sequential Monte Carlo sampler to draw from the importance sampling distribution, which is particularly suited for parallel implementation. With both mathematical and practical examples, we demonstrate the competitive performance of the proposed method.

engineering, industrial,operations research & management science
Single-ensemble multilevel Monte Carlo for discrete interacting-particle methods

Arne Bouillon,Toon Ingelaere,Giovanni Samaey

2024-05-16

Abstract:To solve problems in domains such as filtering, optimization, and posterior sampling, interacting-particle methods have recently received much attention. These parallelizable and often gradient-free algorithms use an ensemble of particles that evolve in time, based on a combination of well-chosen dynamics and interaction between the particles. For computationally expensive dynamics -- for example, dynamics that solve inverse problems with an expensive forward model -- the cost of attaining a high accuracy quickly becomes prohibitive. We exploit a hierarchy of approximations to this forward model and apply multilevel Monte Carlo (MLMC) techniques, improving the asymptotic cost-to-error relation. More specifically, we use MLMC at each time step to estimate the interaction term within a single, globally-coupled ensemble. This technique was proposed by Hoel et al. in the context of the ensemble Kalman filter; the goal of the present paper is to study its applicability to a general framework of interacting-particle methods. After extending the algorithm and its analysis to a broad set of methods with fixed numbers of time steps, we motivate the application of the method to the class of algorithms with an infinite time horizon, which includes popular methods such as ensemble Kalman algorithms for optimization and sampling. Numerical tests confirm the improved asymptotic scaling of the multilevel approach.

Numerical Analysis
Sequential Monte Carlo samplers: error bounds and insensitivity to initial conditions

Nick Whiteley

DOI: https://doi.org/10.48550/arXiv.1103.3970

2011-03-21

Abstract:This paper addresses finite sample stability properties of sequential Monte Carlo methods for approximating sequences of probability distributions. The results presented herein are applicable in the scenario where the start and end distributions in the sequence are fixed and the number of intermediate steps is a parameter of the algorithm. Under assumptions which hold on non-compact spaces, it is shown that the effect of the initial distribution decays exponentially fast in the number of intermediate steps and the corresponding stochastic error is stable in \mathbb{L}_{p} norm.

Computation,Probability
On sequential Monte Carlo, partial rejection control and approximate Bayesian computation

G. W. Peters,Y. Fan,S. A. Sisson

DOI: https://doi.org/10.1007/s11222-012-9315-y

IF: 2.3241

2012-02-15

Statistics and Computing

Abstract:We present a variant of the sequential Monte Carlo sampler by incorporating the partial rejection control mechanism of Liu (2001). We show that the resulting algorithm can be considered as a sequential Monte Carlo sampler with a modified mutation kernel. We prove that the new sampler can reduce the variance of the incremental importance weights when compared with standard sequential Monte Carlo samplers, and provide a central limit theorem. Finally, the sampler is adapted for application under the challenging approximate Bayesian computation modelling framework.

statistics & probability,computer science, theory & methods
Simulation based sequential Monte Carlo methods for discretely observed Markov processes

Peter Neal

DOI: https://doi.org/10.48550/arXiv.1404.4185

2014-04-16

Abstract:Parameter estimation for discretely observed Markov processes is a challenging problem. However, simulation of Markov processes is straightforward using the Gillespie algorithm. We exploit this ease of simulation to develop an effective sequential Monte Carlo (SMC) algorithm for obtaining samples from the posterior distribution of the parameters. In particular, we introduce two key innovations, coupled simulations, which allow us to study multiple parameter values on the basis of a single simulation, and a simple, yet effective, importance sampling scheme for steering simulations towards the observed data. These innovations substantially improve the efficiency of the SMC algorithm with minimal effect on the speed of the simulation process. The SMC algorithm is successfully applied to two examples, a Lotka-Volterra model and a Repressilator model.

Computation
Ensemble Kalman filter based sequential Monte Carlo sampler for sequential Bayesian inference

Jiangqi Wu,Linjie Wen,Peter L. Green,Jinglai Li,Simon Maskell

DOI: https://doi.org/10.1007/s11222-021-10075-x

IF: 2.3241

2022-02-15

Statistics and Computing

Abstract:Abstract Many real-world problems require one to estimate parameters of interest, in a Bayesian framework, from data that are collected sequentially in time. Conventional methods for sampling from posterior distributions, such as Markov chain Monte Carlo cannot efficiently address such problems as they do not take advantage of the data’s sequential structure. To this end, sequential methods which seek to update the posterior distribution whenever a new collection of data become available are often used to solve these types of problems. Two popular choices of sequential method are the ensemble Kalman filter (EnKF) and the sequential Monte Carlo sampler (SMCS). While EnKF only computes a Gaussian approximation of the posterior distribution, SMCS can draw samples directly from the posterior. Its performance, however, depends critically upon the kernels that are used. In this work, we present a method that constructs the kernels of SMCS using an EnKF formulation, and we demonstrate the performance of the method with numerical examples.

statistics & probability,computer science, theory & methods
Multilevel Sequential${}^2$ Monte Carlo for Bayesian Inverse Problems

Jonas Latz,Iason Papaioannou,Elisabeth Ullmann

DOI: https://doi.org/10.1016/j.jcp.2018.04.014

2018-05-06

Abstract:The identification of parameters in mathematical models using noisy observations is a common task in uncertainty quantification. We employ the framework of Bayesian inversion: we combine monitoring and observational data with prior information to estimate the posterior distribution of a parameter. Specifically, we are interested in the distribution of a diffusion coefficient of an elliptic PDE. In this setting, the sample space is high-dimensional, and each sample of the PDE solution is expensive. To address these issues we propose and analyse a novel Sequential Monte Carlo (SMC) sampler for the approximation of the posterior distribution. Classical, single-level SMC constructs a sequence of measures, starting with the prior distribution, and finishing with the posterior distribution. The intermediate measures arise from a tempering of the likelihood, or, equivalently, a rescaling of the noise. The resolution of the PDE discretisation is fixed. In contrast, our estimator employs a hierarchy of PDE discretisations to decrease the computational cost. We construct a sequence of intermediate measures by decreasing the temperature or by increasing the discretisation level at the same time. This idea builds on and generalises the multi-resolution sampler proposed in [P.S. Koutsourelakis, J. Comput. Phys., 228 (2009), pp. 6184-6211] where a bridging scheme is used to transfer samples from coarse to fine discretisation levels. Importantly, our choice between tempering and bridging is fully adaptive. We present numerical experiments in 2D space, comparing our estimator to single-level SMC and the multi-resolution sampler.

Computation
Inference with Hamiltonian Sequential Monte Carlo Simulators

Remi Daviet

DOI: https://doi.org/10.48550/arXiv.1812.07978

2018-12-19

Computation

Abstract:The paper proposes a new Monte-Carlo simulator combining the advantages of Sequential Monte Carlo simulators and Hamiltonian Monte Carlo simulators. The result is a method that is robust to multimodality and complex shapes to use for inference in presence of difficult likelihoods or target functions. Several examples are provided.
Stochastic Gradient Piecewise Deterministic Monte Carlo Samplers

Paul Fearnhead,Sebastiano Grazzi,Chris Nemeth,Gareth O. Roberts

2024-06-27

Abstract:Recent work has suggested using Monte Carlo methods based on piecewise deterministic Markov processes (PDMPs) to sample from target distributions of interest. PDMPs are non-reversible continuous-time processes endowed with momentum, and hence can mix better than standard reversible MCMC samplers. Furthermore, they can incorporate exact sub-sampling schemes which only require access to a single (randomly selected) data point at each iteration, yet without introducing bias to the algorithm's stationary distribution. However, the range of models for which PDMPs can be used, particularly with sub-sampling, is limited. We propose approximate simulation of PDMPs with sub-sampling for scalable sampling from posterior distributions. The approximation takes the form of an Euler approximation to the true PDMP dynamics, and involves using an estimate of the gradient of the log-posterior based on a data sub-sample. We thus call this class of algorithms stochastic-gradient PDMPs. Importantly, the trajectories of stochastic-gradient PDMPs are continuous and can leverage recent ideas for sampling from measures with continuous and atomic components. We show these methods are easy to implement, present results on their approximation error and demonstrate numerically that this class of algorithms has similar efficiency to, but is more robust than, stochastic gradient Langevin dynamics.

Machine Learning,Computation
Sequential Monte Carlo for cut-Bayesian posterior computation

Gopalan, Giri

DOI: https://doi.org/10.1007/s00180-024-01576-0

IF: 1.4049

2024-12-05

Computational Statistics

Abstract:We propose a sequential Monte Carlo (SMC) method to efficiently and accurately compute cut-Bayesian posterior quantities of interest, variations of standard Bayesian approaches constructed primarily to account for model misspecification. We prove finite sample concentration bounds for estimators derived from the proposed method and apply these results to a realistic setting where a computer model is misspecified. Two theoretically justified variations are presented for making the sequential Monte Carlo estimator more computationally efficient, based on linear tempering and finding suitable permutations of initial parameter draws. We then illustrate the SMC method for inference in a modular chemical reactor example that includes submodels for reaction kinetics, turbulence, mass transfer, and diffusion. The samples obtained are commensurate with a direct-sampling approach that consists of running multiple Markov chains, with computational efficiency gains using the SMC method. Overall, the SMC method presented yields a novel, rigorous approach to computing with cut-Bayesian posterior distributions.

statistics & probability
Stratification and Optimal Resampling for Sequential Monte Carlo

Yichao Li,Wenshuo Wang,K. E. Deng,Jun S. Liu

DOI: https://doi.org/10.1093/biomet/asab004

IF: 3.0279

2022-01-01

Biometrika

Abstract:Summary Sequential Monte Carlo algorithms are widely accepted as powerful computational tools for making inference with dynamical systems. A key step in sequential Monte Carlo is resampling, which plays the role of steering the algorithm towards the future dynamics. Several strategies have been used in practice, including multinomial resampling, residual resampling, optimal resampling, stratified resampling and optimal transport resampling. In one-dimensional cases, we show that optimal transport resampling is equivalent to stratified resampling on the sorted particles, and both strategies minimize the resampling variance as well as the expected squared energy distance between the original and resampled empirical distributions. For general $d$-dimensional cases, we show that if the particles are first sorted using the Hilbert curve, the variance of stratified resampling is $O(m^{-(1+2/d)})$, an improvement over the best previously known rate of $O(m^{-(1+1/d)})$, where $m$ is the number of resampled particles. We show that this improved rate is optimal for ordered stratified resampling schemes, as conjectured in Gerber et al. (2019). We also present an almost-sure bound on the Wasserstein distance between the original and Hilbert-curve-resampled empirical distributions. In light of these results, we show that for dimension $d>1$ the mean square error of sequential quasi-Monte Carlo with $n$ particles can be $O(n^{-1-4/\{d(d+4)\}})$ if Hilbert curve resampling is used and a specific low-discrepancy set is chosen. To our knowledge, this is the first known convergence rate lower than $o(n^{-1})$.
The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method

Alexandre Bouchard-Côté,Sebastian J. Vollmer,Arnaud Doucet

DOI: https://doi.org/10.1080/01621459.2017.1294075

IF: 4.369

2018-04-03

Journal of the American Statistical Association

Abstract:Many Markov chain Monte Carlo techniques currently available rely on discrete-time reversible Markov processes whose transition kernels are variations of the Metropolis–Hastings algorithm. We explore and generalize an alternative scheme recently introduced in the physics literature (Peters and de With 2012) where the target distribution is explored using a continuous-time nonreversible piecewise-deterministic Markov process. In the Metropolis–Hastings algorithm, a trial move to a region of lower target density, equivalently of higher “energy,” than the current state can be rejected with positive probability. In this alternative approach, a particle moves along straight lines around the space and, when facing a high energy barrier, it is not rejected but its path is modified by bouncing against this barrier. By reformulating this algorithm using inhomogeneous Poisson processes, we exploit standard sampling techniques to simulate exactly this Markov process in a wide range of scenarios of interest. Additionally, when the target distribution is given by a product of factors dependent only on subsets of the state variables, such as the posterior distribution associated with a probabilistic graphical model, this method can be modified to take advantage of this structure by allowing computationally cheaper “local” bounces, which only involve the state variables associated with a factor, while the other state variables keep on evolving. In this context, by leveraging techniques from chemical kinetics, we propose several computationally efficient implementations. Experimentally, this new class of Markov chain Monte Carlo schemes compares favorably to state-of-the-art methods on various Bayesian inference tasks, including for high-dimensional models and large datasets. Supplementary materials for this article are available online.

statistics & probability
Waste-free Sequential Monte Carlo

Hai-Dang Dau,Nicolas Chopin

DOI: https://doi.org/10.48550/arXiv.2011.02328

2021-08-23

Abstract:A standard way to move particles in a SMC sampler is to apply several steps of a MCMC (Markov chain Monte Carlo) kernel. Unfortunately, it is not clear how many steps need to be performed for optimal performance. In addition, the output of the intermediate steps are discarded and thus wasted somehow. We propose a new, waste-free SMC algorithm which uses the outputs of all these intermediate MCMC steps as particles. We establish that its output is consistent and asymptotically normal. We use the expression of the asymptotic variance to develop various insights on how to implement the algorithm in practice. We develop in particular a method to estimate, from a single run of the algorithm, the asymptotic variance of any particle estimate. We show empirically, through a range of numerical examples, that waste-free SMC tends to outperform standard SMC samplers, and especially so in situations where the mixing of the considered MCMC kernels decreases across iterations (as in tempering or rare event problems).

Computation
Persistent Sampling: Unleashing the Potential of Sequential Monte Carlo

Minas Karamanis,Uroš Seljak

2024-07-30

Abstract:Sequential Monte Carlo (SMC) methods are powerful tools for Bayesian inference but suffer from requiring many particles for accurate estimates, leading to high computational costs. We introduce persistent sampling (PS), an extension of SMC that mitigates this issue by allowing particles from previous iterations to persist. This generates a growing, weighted ensemble of particles distributed across iterations. In each iteration, PS utilizes multiple importance sampling and resampling from the mixture of all previous distributions to produce the next generation of particles. This addresses particle impoverishment and mode collapse, resulting in more accurate posterior approximations. Furthermore, this approach provides lower-variance marginal likelihood estimates for model comparison. Additionally, the persistent particles improve transition kernel adaptation for efficient exploration. Experiments on complex distributions show that PS consistently outperforms standard methods, achieving lower squared bias in posterior moment estimation and significantly reduced marginal likelihood errors, all at a lower computational cost. PS offers a robust, efficient, and scalable framework for Bayesian inference.

Machine Learning,Computation
Elements of Sequential Monte Carlo

Christian A. Naesseth,Fredrik Lindsten,Thomas B. Schön

DOI: https://doi.org/10.48550/arXiv.1903.04797

IF: 5.414

2019-03-12

Machine Learning

Abstract:A core problem in statistics and probabilistic machine learning is to compute probability distributions and expectations. This is the fundamental problem of Bayesian statistics and machine learning, which frames all inference as expectations with respect to the posterior distribution. The key challenge is to approximate these intractable expectations. In this tutorial, we review sequential Monte Carlo (SMC), a random-sampling-based class of methods for approximate inference. First, we explain the basics of SMC, discuss practical issues, and review theoretical results. We then examine two of the main user design choices: the proposal distributions and the so called intermediate target distributions. We review recent results on how variational inference and amortization can be used to learn efficient proposals and target distributions. Next, we discuss the SMC estimate of the normalizing constant, how this can be used for pseudo-marginal inference and inference evaluation. Throughout the tutorial we illustrate the use of SMC on various models commonly used in machine learning, such as stochastic recurrent neural networks, probabilistic graphical models, and probabilistic programs.

Sequential monte carlo samplers

An invitation to sequential Monte Carlo samplers

Sequential Monte Carlo with Highly Informative Observations

An Introduction to Sequential Monte Carlo

On sequential Monte Carlo sampling methods for Bayesian filtering

Convergence of sequential Markov Chain Monte Carlo methods: I. Nonlinear flow of probability measures

Multicanonical sequential Monte Carlo sampler for uncertainty quantification

Single-ensemble multilevel Monte Carlo for discrete interacting-particle methods

Sequential Monte Carlo samplers: error bounds and insensitivity to initial conditions

On sequential Monte Carlo, partial rejection control and approximate Bayesian computation

Simulation based sequential Monte Carlo methods for discretely observed Markov processes

Ensemble Kalman filter based sequential Monte Carlo sampler for sequential Bayesian inference

Multilevel Sequential${}^2$ Monte Carlo for Bayesian Inverse Problems

Inference with Hamiltonian Sequential Monte Carlo Simulators

Stochastic Gradient Piecewise Deterministic Monte Carlo Samplers

Sequential Monte Carlo for cut-Bayesian posterior computation

Stratification and Optimal Resampling for Sequential Monte Carlo

The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method

Waste-free Sequential Monte Carlo

Persistent Sampling: Unleashing the Potential of Sequential Monte Carlo

Elements of Sequential Monte Carlo