Abstract:Models of stochastic processes are widely used in almost all fields of science. Theory validation, parameter estimation, and prediction all require model calibration and statistical inference using data. However, data are almost always incomplete observations of reality. This leads to a great challenge for statistical inference because the likelihood function will be intractable for almost all partially observed stochastic processes. This renders many statistical methods, especially within a Bayesian framework, impossible to implement. Therefore, computationally expensive likelihood-free approaches are applied that replace likelihood evaluations with realisations of the model and observation process. For accurate inference, however, likelihood-free techniques may require millions of expensive stochastic simulations. To address this challenge, we develop a new method based on recent advances in multilevel and multifidelity. Our approach combines the multilevel Monte Carlo telescoping summation, applied to a sequence of approximate Bayesian posterior targets, with a multifidelity rejection sampler to minimise the number of computationally expensive exact simulations required for accurate inference. We present the derivation of our new algorithm for likelihood-free Bayesian inference, discuss practical implementation details, and demonstrate substantial performance improvements. Using examples from systems biology, we demonstrate improvements of more than two orders of magnitude over standard rejection sampling techniques. Our approach is generally applicable to accelerate other sampling schemes, such as sequential Monte Carlo, to enable feasible Bayesian analysis for realistic practical applications in physics, chemistry, biology, epidemiology, ecology and economics.

A review of deterministic approximate inference techniques for Bayesian machine learning

Non-linear regression models for Approximate Bayesian Computation

Understanding Approximation for Bayesian Inference in Neural Networks

An effective likelihood-free approximate computing method with statistical inferential guarantees

An automatic adaptive method to combine summary statistics in approximate Bayesian computation

Approximate inference for dynamic Bayesian networks: sliding window approach

Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions

A Learning Error Analysis for Structured Prediction with Approximate Inference

Approximate Methods for Bayesian Computation

Generalized Posteriors in Approximate Bayesian Computation

Approximating multivariate posterior distribution functions from Monte Carlo samples for sequential Bayesian inference

Approximate Bayesian Computation for Smoothing

A rare event approach to high dimensional Approximate Bayesian computation

Variational Inference: A Review for Statisticians

A review of predictive uncertainty estimation with machine learning

ABC-CDE: Towards Approximate Bayesian Computation with Complex High-Dimensional Data and Limited Simulations

A comparison of algorithms for inference and learning in probabilistic graphical models

Multifidelity multilevel Monte Carlo to accelerate approximate Bayesian parameter inference for partially observed stochastic processes

Correcting Predictions for Approximate Bayesian Inference

AIDE: An algorithm for measuring the accuracy of probabilistic inference algorithms

Inference for SDE models via Approximate Bayesian Computation