Abstract:Many models of interest in the natural and social sciences have no closed-form likelihood function, which means that they cannot be treated using the usual techniques of statistical inference. In the case where such models can be efficiently simulated, Bayesian inference is still possible thanks to the Approximate Bayesian Computation (ABC) algorithm. Although many refinements have been suggested, ABC inference is still far from routine. ABC is often excruciatingly slow due to very low acceptance rates. In addition, ABC requires introducing a vector of "summary statistics", the choice of which is relatively arbitrary, and often require some trial and error, making the whole process quite laborious for the user. We introduce in this work the EP-ABC algorithm, which is an adaptation to the likelihood-free context of the variational approximation algorithm known as Expectation Propagation (Minka, 2001). The main advantage of EP-ABC is that it is faster by a few orders of magnitude than standard algorithms, while producing an overall approximation error which is typically negligible. A second advantage of EP-ABC is that it replaces the usual global ABC constraint on the vector of summary statistics computed on the whole dataset, by n local constraints of the form that apply separately to each data-point. As a consequence, it is often possible to do away with summary statistics entirely. In that case, EP-ABC approximates directly the evidence (marginal likelihood) of the model. Comparisons are performed in three real-world applications which are typical of likelihood-free inference, including one application in neuroscience which is novel, and possibly too challenging for standard ABC techniques.

Expectation Propagation for Estimating the Parameters of the Beta Distribution

Bayesian Estimation of the Dirichlet Distribution with Expectation Propagation.

Approximating the predictive distribution of the beta distribution with the local variational method

Bayesian Estimation of Beta Mixture Models with Variational Inference

Approximate Posterior Inference for Bayesian Models: Black-Box Expectation Propagation

Expectation Propagation in the large-data limit

Generalizing Expectation Propagation with Mixtures of Exponential Family Distributions and an Application to Bayesian Logistic Regression.

Expectation Propagation for approximate Bayesian inference

Bayesian Inference for the Beta-Binomial Distribution via Polynomial Expansions

Extended Variational Inference for Dirichlet Process Mixture of Beta‐Liouville Distributions for Proportional Data Modeling

Expectation-Propagation for Likelihood-Free Inference

Bayesian parameter estimation via variational methods

Expectation propagation for Poisson data

Variational Learning for Finite Beta-Liouville Mixture Models and Its Application

Upper Expectation Parametric Regression

EMPIRICAL BAYESIAN ESTIMATION OF THE LOCATION PARAMETER IN DOUBLE EXPONENTIAL DISTRIBUTION

Probabilistic Inference of Bayesian Neural Networks with Generalized Expectation Propagation

Bayesian Estimation Of Beta Distribution

Estimating Bayesian Networks Parameters Using EM and Gibbs Sampling

Expectation Propogation for approximate inference in dynamic Bayesian networks

Fearless Stochasticity in Expectation Propagation