Overview of Approximate Bayesian Computation

S. A. Sisson,Y. Fan,M. A. Beaumont

DOI: https://doi.org/10.48550/arXiv.1802.09720

2018-02-27

Abstract:This Chapter, "Overview of Approximate Bayesian Computation", is to appear as the first chapter in the forthcoming Handbook of Approximate Bayesian Computation (2018). It details the main ideas and concepts behind ABC methods with many examples and illustrations.

Computation,Methodology,Machine Learning

What problem does this paper attempt to address?

Summary Statistics in Approximate Bayesian Computation

Dennis Prangle

DOI: https://doi.org/10.48550/arXiv.1512.05633

2015-12-17

Abstract:This document is due to appear as a chapter of the forthcoming Handbook of Approximate Bayesian Computation (ABC) edited by S. Sisson, Y. Fan, and M. Beaumont. Since the earliest work on ABC, it has been recognised that using summary statistics is essential to produce useful inference results. This is because ABC suffers from a curse of dimensionality effect, whereby using high dimensional inputs causes large approximation errors in the output. It is therefore crucial to find low dimensional summaries which are informative about the parameter inference or model choice task at hand. This chapter reviews the methods which have been proposed to select such summaries, extending the previous review paper of Blum et al. (2013) with recent developments. Related theoretical results on the ABC curse of dimensionality and sufficiency are also discussed.

Computation,Statistics Theory,Methodology
Approximate Bayesian Forecasting

David T. Frazier,Worapree Maneesoonthorn,Gael M. Martin,Brendan P.M. McCabe

DOI: https://doi.org/10.48550/arXiv.1712.07750

2018-06-20

Abstract:Approximate Bayesian Computation (ABC) has become increasingly prominent as a method for conducting parameter inference in a range of challenging statistical problems, most notably those characterized by an intractable likelihood function. In this paper, we focus on the use of ABC not as a tool for parametric inference, but as a means of generating probabilistic forecasts; or for conducting what we refer to as `approximate Bayesian forecasting'. The four key issues explored are: i) the link between the theoretical behavior of the ABC posterior and that of the ABC-based predictive; ii) the use of proper scoring rules to measure the (potential) loss of forecast accuracy when using an approximate rather than an exact predictive; iii) the performance of approximate Bayesian forecasting in state space models; and iv) the use of forecasting criteria to inform the selection of ABC summaries in empirical settings. The primary finding of the paper is that ABC can provide a computationally efficient means of generating probabilistic forecasts that are nearly identical to those produced by the exact predictive, and in a fraction of the time required to produce predictions via an exact method. y identical to those produced by the exact predictive, and in a fraction of the time required to produce predictions via an exact method.

Methodology
Approximate Bayesian Computation for Smoothing

James S. Martin,Ajay Jasra,Sumeetpal S. Singh,Nick Whiteley,Emma McCoy

DOI: https://doi.org/10.48550/arXiv.1206.5208

2012-06-23

Abstract:We consider a method for approximate inference in hidden Markov models (HMMs). The method circumvents the need to evaluate conditional densities of observations given the hidden states. It may be considered an instance of Approximate Bayesian Computation (ABC) and it involves the introduction of auxiliary variables valued in the same space as the observations. The quality of the approximation may be controlled to arbitrary precision through a parameter \epsilon>0 . We provide theoretical results which quantify, in terms of \epsilon, the ABC error in approximation of expectations of additive functionals with respect to the smoothing distributions. Under regularity assumptions, this error is O(n\epsilon), where n is the number of time steps over which smoothing is performed. For numerical implementation we adopt the forward-only sequential Monte Carlo (SMC) scheme of [16] and quantify the combined error from the ABC and SMC approximations. This forms some of the first quantitative results for ABC methods which jointly treat the ABC and simulation errors, with a finite number of data and simulated samples. When the HMM has unknown static parameters, we consider particle Markov chain Monte Carlo [2] (PMCMC) methods for batch statistical inference.

Computation,Methodology
High-dimensional ABC

D. J. Nott,V. M.-H. Ong,Y. Fan,S. A. Sisson

DOI: https://doi.org/10.48550/arXiv.1802.09725

2018-02-27

Abstract:This Chapter, "High-dimensional ABC", is to appear in the forthcoming Handbook of Approximate Bayesian Computation (2018). It details the main ideas and concepts behind extending ABC methods to higher dimensions, with supporting examples and illustrations.

Computation,Methodology,Machine Learning
ABC Samplers

Y. Fan,S. A. Sisson

DOI: https://doi.org/10.48550/arXiv.1802.09650

2018-02-27

Abstract:This Chapter, "ABC Samplers", is to appear in the forthcoming Handbook of Approximate Bayesian Computation (2018). It details the main ideas and algorithms used to sample from the ABC approximation to the posterior distribution, including methods based on rejection/importance sampling, MCMC and sequential Monte Carlo.

Computation,Methodology,Machine Learning
Approximate Bayesian Computation via Classification

Yuexi Wang,Tetsuya Kaji,Veronika Ročková

DOI: https://doi.org/10.48550/arXiv.2111.11507

2022-12-01

Abstract:Approximate Bayesian Computation (ABC) enables statistical inference in simulator-based models whose likelihoods are difficult to calculate but easy to simulate from. ABC constructs a kernel-type approximation to the posterior distribution through an accept/reject mechanism which compares summary statistics of real and simulated data. To obviate the need for summary statistics, we directly compare empirical distributions with a Kullback-Leibler (KL) divergence estimator obtained via contrastive learning. In particular, we blend flexible machine learning classifiers within ABC to automate fake/real data comparisons. We consider the traditional accept/reject kernel as well as an exponential weighting scheme which does not require the ABC acceptance threshold. Our theoretical results show that the rate at which our ABC posterior distributions concentrate around the true parameter depends on the estimation error of the classifier. We derive limiting posterior shape results and find that, with a properly scaled exponential kernel, asymptotic normality holds. We demonstrate the usefulness of our approach on simulated examples as well as real data in the context of stock volatility estimation.

Methodology,Machine Learning
ABC-CDE: Towards Approximate Bayesian Computation with Complex High-Dimensional Data and Limited Simulations

Rafael Izbicki,Ann B. Lee,Taylor Pospisil

DOI: https://doi.org/10.1080/10618600.2018.1546594

2018-10-20

Abstract:Approximate Bayesian Computation (ABC) is typically used when the likelihood is either unavailable or intractable but where data can be simulated under different parameter settings using a forward model. Despite the recent interest in ABC, high-dimensional data and costly simulations still remain a bottleneck in some applications. There is also no consensus as to how to best assess the performance of such methods without knowing the true posterior. We show how a nonparametric conditional density estimation (CDE) framework, which we refer to as ABC-CDE, help address three nontrivial challenges in ABC: (i) how to efficiently estimate the posterior distribution with limited simulations and different types of data, (ii) how to tune and compare the performance of ABC and related methods in estimating the posterior itself, rather than just certain properties of the density, and (iii) how to efficiently choose among a large set of summary statistics based on a CDE surrogate loss. We provide theoretical and empirical evidence that justify ABC-CDE procedures that {\em directly} estimate and assess the posterior based on an initial ABC sample, and we describe settings where standard ABC and regression-based approaches are inadequate.

Methodology,Machine Learning
Approximating the Likelihood in Approximate Bayesian Computation

Christopher C Drovandi,Clara Grazian,Kerrie Mengersen,Christian Robert

DOI: https://doi.org/10.48550/arXiv.1803.06645

2018-03-18

Abstract:This chapter will appear in the forthcoming Handbook of Approximate Bayesian Computation (2018). The conceptual and methodological framework that underpins approximate Bayesian computation (ABC) is targetted primarily towards problems in which the likelihood is either challenging or missing. ABC uses a simulation-based non-parametric estimate of the likelihood of a summary statistic and assumes that the generation of data from the model is computationally cheap. This chapter reviews two alternative approaches for estimating the intractable likelihood, with the goal of reducing the necessary model simulations to produce an approximate posterior. The first of these is a Bayesian version of the synthetic likelihood (SL), initially developed by Wood (2010), which uses a multivariate normal approximation to the summary statistic likelihood. Using the parametric approximation as opposed to the non-parametric approximation of ABC, it is possible to reduce the number of model simulations required. The second likelihood approximation method we consider in this chapter is based on the empirical likelihood (EL), which is a non-parametric technique and involves maximising a likelihood constructed empirically under a set of moment constraints. Mengersen et al (2013) adapt the EL framework so that it can be used to form an approximate posterior for problems where ABC can be applied, that is, for models with intractable likelihoods. However, unlike ABC and the Bayesian SL (BSL), the Bayesian EL (BCel) approach can be used to completely avoid model simulations in some cases. The BSL and BCel methods are illustrated on models of varying complexity.

Computation,Methodology
Approximate Bayesian Computation for a Class of Time Series Models

Ajay Jasra

DOI: https://doi.org/10.48550/arXiv.1401.0265

2014-01-01

Abstract:In the following article we consider approximate Bayesian computation (ABC) for certain classes of time series models. In particular, we focus upon scenarios where the likelihoods of the observations and parameter are intractable, by which we mean that one cannot evaluate the likelihood even up-to a positive unbiased estimate. This paper reviews and develops a class of approximation procedures based upon the idea of ABC, but, specifically maintains the probabilistic structure of the original statistical model. This idea is useful, in that it can facilitate an analysis of the bias of the approximation and the adaptation of established computational methods for parameter inference. Several existing results in the literature are surveyed and novel developments with regards to computation are given.

Computation
Approximate Bayesian Computation via Regression Density Estimation

Y. Fan,D. J. Nott,S. A. Sisson

DOI: https://doi.org/10.48550/arXiv.1212.1479

2012-12-07

Abstract:Approximate Bayesian computation (ABC) methods, which are applicable when the likelihood is difficult or impossible to calculate, are an active topic of current research. Most current ABC algorithms directly approximate the posterior distribution, but an alternative, less common strategy is to approximate the likelihood function. This has several advantages. First, in some problems, it is easier to approximate the likelihood than to approximate the posterior. Second, an approximation to the likelihood allows reference analyses to be constructed based solely on the likelihood. Third, it is straightforward to perform sensitivity analyses for several different choices of prior once an approximation to the likelihood is constructed, which needs to be done only once. The contribution of the present paper is to consider regression density estimation techniques to approximate the likelihood in the ABC setting. Our likelihood approximations build on recently developed marginal adaptation density estimators by extending them for conditional density estimation. Our approach facilitates reference Bayesian inference, as well as frequentist inference. The method is demonstrated via a challenging problem of inference for stereological extremes, where we perform both frequentist and Bayesian inference.

Computation,Methodology
An automatic adaptive method to combine summary statistics in approximate Bayesian computation

Jonathan U Harrison,Ruth E Baker

DOI: https://doi.org/10.48550/arXiv.1703.02341

2018-08-02

Abstract:To infer the parameters of mechanistic models with intractable likelihoods, techniques such as approximate Bayesian computation (ABC) are increasingly being adopted. One of the main disadvantages of ABC in practical situations, however, is that parameter inference must generally rely on summary statistics of the data. This is particularly the case for problems involving high-dimensional data, such as biological imaging experiments. However, some summary statistics contain more information about parameters of interest than others, and it is not always clear how to weight their contributions within the ABC framework. We address this problem by developing an automatic, adaptive algorithm that chooses weights for each summary statistic. Our algorithm aims to maximize the distance between the prior and the approximate posterior by automatically adapting the weights within the ABC distance function. Computationally, we use a nearest neighbour estimator of the distance between distributions. We justify the algorithm theoretically based on properties of the nearest neighbour distance estimator. To demonstrate the effectiveness of our algorithm, we apply it to a variety of test problems, including several stochastic models of biochemical reaction networks, and a spatial model of diffusion, and compare our results with existing algorithms.

Computation
An Approximate Likelihood Perspective on ABC Methods

George Karabatsos,Fabrizio Leisen

DOI: https://doi.org/10.48550/arXiv.1708.05341

2018-05-08

Abstract:We are living in the big data era, as current technologies and networks allow for the easy and routine collection of data sets in different disciplines. Bayesian Statistics offers a flexible modeling approach which is attractive for describing the complexity of these datasets. These models often exhibit a likelihood function which is intractable due to the large sample size, high number of parameters, or functional complexity. Approximate Bayesian Computational (ABC) methods provides likelihood-free methods for performing statistical inferences with Bayesian models defined by intractable likelihood functions. The vastity of the literature on ABC methods created a need to review and relate all ABC approaches so that scientists can more readily understand and apply them for their own work. This article provides a unifying review, general representation, and classification of all ABC methods from the view of approximate likelihood theory. This clarifies how ABC methods can be characterized, related, combined, improved, and applied for future research. Possible future research in ABC is then suggested.

Methodology
Generalized Posteriors in Approximate Bayesian Computation

Sebastian M Schmon,Patrick W Cannon,Jeremias Knoblauch

DOI: https://doi.org/10.48550/arXiv.2011.08644

2021-02-24

Abstract:Complex simulators have become a ubiquitous tool in many scientific disciplines, providing high-fidelity, implicit probabilistic models of natural and social phenomena. Unfortunately, they typically lack the tractability required for conventional statistical analysis. Approximate Bayesian computation (ABC) has emerged as a key method in simulation-based inference, wherein the true model likelihood and posterior are approximated using samples from the simulator. In this paper, we draw connections between ABC and generalized Bayesian inference (GBI). First, we re-interpret the accept/reject step in ABC as an implicitly defined error model. We then argue that these implicit error models will invariably be misspecified. While ABC posteriors are often treated as a necessary evil for approximating the standard Bayesian posterior, this allows us to re-interpret ABC as a potential robustification strategy. This leads us to suggest the use of GBI within ABC, a use case we explore empirically.

Methodology,Computation,Machine Learning
Asymptotic Behaviour of Approximate Bayesian Estimators

Thomas A. Dean,Sumeetpal S. Singh

DOI: https://doi.org/10.48550/arXiv.1105.3655

2011-05-18

Abstract:Although approximate Bayesian computation (ABC) has become a popular technique for performing parameter estimation when the likelihood functions are analytically intractable there has not as yet been a complete investigation of the theoretical properties of the resulting estimators. In this paper we give a theoretical analysis of the asymptotic properties of ABC based parameter estimators for hidden Markov models and show that ABC based estimators satisfy asymptotically biased versions of the standard results in the statistical literature.

Statistics Theory
Approximate Bayesian Computation with the Sliced-Wasserstein Distance

Kimia Nadjahi,Valentin De Bortoli,Alain Durmus,Roland Badeau,Umut Şimşekli

DOI: https://doi.org/10.48550/arXiv.1910.12815

2020-03-06

Abstract:Approximate Bayesian Computation (ABC) is a popular method for approximate inference in generative models with intractable but easy-to-sample likelihood. It constructs an approximate posterior distribution by finding parameters for which the simulated data are close to the observations in terms of summary statistics. These statistics are defined beforehand and might induce a loss of information, which has been shown to deteriorate the quality of the approximation. To overcome this problem, Wasserstein-ABC has been recently proposed, and compares the datasets via the Wasserstein distance between their empirical distributions, but does not scale well to the dimension or the number of samples. We propose a new ABC technique, called Sliced-Wasserstein ABC and based on the Sliced-Wasserstein distance, which has better computational and statistical properties. We derive two theoretical results showing the asymptotical consistency of our approach, and we illustrate its advantages on synthetic data and an image denoising task.

Computation,Methodology,Machine Learning
A comparison of emulation methods for Approximate Bayesian Computation

Franck Jabot,Guillaume Lagarrigues,Benoît Courbaud,Nicolas Dumoulin

DOI: https://doi.org/10.48550/arXiv.1412.7560

2014-12-16

Abstract:Approximate Bayesian Computation (ABC) is a family of statistical inference techniques, which is increasingly used in biology and other scientific fields. Its main benefit is to be applicable to models for which the computation of the model likelihood is intractable. The basic idea of ABC is to empirically approximate the model likelihood by using intensive realizations of model runs. Due to computing time limitations, ABC has thus been mainly applied to models that are relatively quick to simulate. We here aim at briefly introducing the field of statistical emulation of computer code outputs and to demonstrate its potential for ABC applications. Emulation consists in replacing the costly to simulate model by another (quick to simulate) statistical model called emulator or meta-model. This emulator is fitted to a small number of outputs of the original model, and is subsequently used as a surrogate during the inference procedure. In this contribution, we first detail the principles of model emulation, with a special reference to the ABC context in which the description of the stochasticity of model realizations is as important as the description of the trends linking model parameters and outputs. We then compare several emulation strategies in an ABC context, using as case study a stochastic ecological model of community dynamics. We finally describe a novel emulation-based sequential ABC algorithm which is shown to decrease computing time by a factor of two on the studied example, compared to previous sequential ABC algorithms. Routines to perform emulation-based ABC were made available within the R package EasyABC.

Quantitative Methods,Populations and Evolution,Computation
Probably approximate Bayesian computation: nonasymptotic convergence of ABC under misspecification

James Ridgway

DOI: https://doi.org/10.48550/arXiv.1707.05987

2019-01-01

Abstract:Approximate Bayesian computation (ABC) is a widely used inference method in Bayesian statistics to bypass the point-wise computation of the likelihood. In this paper we develop theoretical bounds for the distance between the statistics used in ABC. We show that some versions of ABC are inherently robust to misspecification. The bounds are given in the form of oracle inequalities for a finite sample size. The dependence on the dimension of the parameter space and the number of statistics is made explicit. The results are shown to be amenable to oracle inequalities in parameter space. We apply our theoretical results to given prior distributions and data generating processes, including a non-parametric regression model. In a second part of the paper, we propose a sequential Monte Carlo (SMC) to sample from the pseudo-posterior, improving upon the state of the art samplers.

Statistics Theory,Machine Learning,Computation
On an Empirical Likelihood based Solution to the Approximate Bayesian Computation Problem

Sanjay Chaudhuri,Subhroshekhar Ghosh,Kim Cuc Pham

2024-03-08

Abstract:Approximate Bayesian Computation (ABC) methods are applicable to statistical models specified by generative processes with analytically intractable likelihoods. These methods try to approximate the posterior density of a model parameter by comparing the observed data with additional process-generated simulated datasets. For computational benefit, only the values of certain well-chosen summary statistics are usually compared, instead of the whole dataset. Most ABC procedures are computationally expensive, justified only heuristically, and have poor asymptotic properties. In this article, we introduce a new empirical likelihood-based approach to the ABC paradigm called ABCel. The proposed procedure is computationally tractable and approximates the target log posterior of the parameter as a sum of two functions of the data -- namely, the mean of the optimal log-empirical likelihood weights and the estimated differential entropy of the summary functions. We rigorously justify the procedure via direct and reverse information projections onto appropriate classes of probability densities. Past applications of empirical likelihood in ABC demanded constraints based on analytically tractable estimating functions that involve both the data and the parameter; although by the nature of the ABC problem such functions may not be available in general. In contrast, we use constraints that are functions of the summary statistics only. Equally importantly, we show that our construction directly connects to the reverse information projection. We show that ABCel is posterior consistent and has highly favourable asymptotic properties. Its construction justifies the use of simple summary statistics like moments, quantiles, etc, which in practice produce an accurate approximation of the posterior density. We illustrate the performance of the proposed procedure in a range of applications.

Methodology
Modelling extremes using approximate Bayesian Computation

Robert Erhardt,Scott A. Sisson

DOI: https://doi.org/10.48550/arXiv.1411.1451

2014-11-06

Abstract:By the nature of their construction, many statistical models for extremes result in likelihood functions that are computationally prohibitive to evaluate. This is consequently problematic for the purposes of likelihood-based inference. With a focus on the Bayesian framework, this chapter examines the use of approximate Bayesian computation (ABC) techniques for the fitting and analysis of statistical models for extremes. After introducing the ideas behind ABC algorithms and methods, we demonstrate their application to extremal models in stereology and spatial extremes.

Methodology,Computation
Improving approximate Bayesian computation via quasi-Monte Carlo

Alexander Buchholz,Nicolas Chopin

DOI: https://doi.org/10.48550/arXiv.1710.01057

2018-05-07

Abstract:ABC (approximate Bayesian computation) is a general approach for dealing with models with an intractable likelihood. In this work, we derive ABC algorithms based on QMC (quasi- Monte Carlo) sequences. We show that the resulting ABC estimates have a lower variance than their Monte Carlo counter-parts. We also develop QMC variants of sequential ABC algorithms, which progressively adapt the proposal distribution and the acceptance threshold. We illustrate our QMC approach through several examples taken from the ABC literature. Keywords: Approximate Bayesian computation, Likelihood-free inference, Quasi Monte Carlo, Randomized Quasi-Monte Carlo, Adaptive importance sampling

Computation

Overview of Approximate Bayesian Computation

Summary Statistics in Approximate Bayesian Computation

Approximate Bayesian Forecasting

Approximate Bayesian Computation for Smoothing

High-dimensional ABC

ABC Samplers

Approximate Bayesian Computation via Classification

ABC-CDE: Towards Approximate Bayesian Computation with Complex High-Dimensional Data and Limited Simulations

Approximating the Likelihood in Approximate Bayesian Computation

Approximate Bayesian Computation for a Class of Time Series Models

Approximate Bayesian Computation via Regression Density Estimation

An automatic adaptive method to combine summary statistics in approximate Bayesian computation

An Approximate Likelihood Perspective on ABC Methods

Generalized Posteriors in Approximate Bayesian Computation

Asymptotic Behaviour of Approximate Bayesian Estimators

Approximate Bayesian Computation with the Sliced-Wasserstein Distance

A comparison of emulation methods for Approximate Bayesian Computation

Probably approximate Bayesian computation: nonasymptotic convergence of ABC under misspecification

On an Empirical Likelihood based Solution to the Approximate Bayesian Computation Problem

Modelling extremes using approximate Bayesian Computation

Improving approximate Bayesian computation via quasi-Monte Carlo