Abstract:Many models of interest in the natural and social sciences have no closed-form likelihood function, which means that they cannot be treated using the usual techniques of statistical inference. In the case where such models can be efficiently simulated, Bayesian inference is still possible thanks to the Approximate Bayesian Computation (ABC) algorithm. Although many refinements have been suggested, ABC inference is still far from routine. ABC is often excruciatingly slow due to very low acceptance rates. In addition, ABC requires introducing a vector of "summary statistics", the choice of which is relatively arbitrary, and often require some trial and error, making the whole process quite laborious for the user. We introduce in this work the EP-ABC algorithm, which is an adaptation to the likelihood-free context of the variational approximation algorithm known as Expectation Propagation (Minka, 2001). The main advantage of EP-ABC is that it is faster by a few orders of magnitude than standard algorithms, while producing an overall approximation error which is typically negligible. A second advantage of EP-ABC is that it replaces the usual global ABC constraint on the vector of summary statistics computed on the whole dataset, by n local constraints of the form that apply separately to each data-point. As a consequence, it is often possible to do away with summary statistics entirely. In that case, EP-ABC approximates directly the evidence (marginal likelihood) of the model. Comparisons are performed in three real-world applications which are typical of likelihood-free inference, including one application in neuroscience which is novel, and possibly too challenging for standard ABC techniques.

Fast Convergent Algorithms for Expectation Propagation Approximate Bayesian Inference

An Expectation Propagation Perspective on Approximate Message Passing

Generalizing Expectation Propagation with Mixtures of Exponential Family Distributions and an Application to Bayesian Logistic Regression.

Expectation Propogation for approximate inference in dynamic Bayesian networks

Expectation Propagation in the large-data limit

Approximate Posterior Inference for Bayesian Models: Black-Box Expectation Propagation

Fearless Stochasticity in Expectation Propagation

Expectation Propagation for approximate Bayesian inference

Expectation propagation for continuous time stochastic processes

Expectation-Propagation for Likelihood-Free Inference

Fast convergence of the Expectation Maximization algorithm under a logarithmic Sobolev inequality

Convex Combination Belief Propagation Algorithms

Entropic Matching for Expectation Propagation of Markov Jump Processes

Double-Chain Unscented Expectation Propagation for Inference in Stochastic Dynamical Models of Biological Processes.

Approximate Inference and Constrained Optimization

Message Passing with L1 Penalized KL Minimization

Efficient computation of predictive probabilities in probit models via expectation propagation

Message Passing with Relaxed Moment Matching

A General Framework for Empirical Bayes Estimation in Discrete Linear Exponential Family

Convergence of Expectation-Maximization Algorithm with Mixed-Integer Optimization

Kernel-based Approximate Bayesian Inference for Exponential Family Random Graph Models