Abstract:We study the problem of bounding the posterior distribution of discrete probabilistic programs with unbounded support, loops, and conditioning. Loops pose the main difficulty in this setting: even if exact Bayesian inference is possible, the state of the art requires user-provided loop invariant templates. By contrast, we aim to find guaranteed bounds, which sandwich the true distribution. They are fully automated, applicable to more programs and provide more provable guarantees than approximate sampling-based inference. Since lower bounds can be obtained by unrolling loops, the main challenge is upper bounds, and we attack it in two ways. The first is called residual mass semantics, which is a flat bound based on the residual probability mass of a loop. The approach is simple, efficient, and has provable guarantees. The main novelty of our work is the second approach, called geometric bound semantics. It operates on a novel family of distributions, called eventually geometric distributions (EGDs), and can bound the distribution of loops with a new form of loop invariants called contraction invariants. The invariant synthesis problem reduces to a system of polynomial inequality constraints, which is a decidable problem with automated solvers. If a solution exists, it yields an exponentially decreasing bound on the whole distribution, and can therefore bound moments and tail asymptotics as well, not just probabilities as in the first approach. Both semantics enjoy desirable theoretical properties. In particular, we prove soundness and convergence, i.e. the bounds converge to the exact posterior as loops are unrolled further. On the practical side, we describe Diabolo, a fully-automated implementation of both semantics, and evaluate them on a variety of benchmarks from the literature, demonstrating their general applicability and the utility of the resulting bounds.

Exact Bayesian Inference for Loopy Probabilistic Programs Using Generating Functions

Exact Probabilistic Inference Using Generating Functions

Does a Program Yield the Right Distribution? Verifying Probabilistic Programs via Generating Functions

Guaranteed Bounds on Posterior Distributions of Discrete Probabilistic Programs with Loops

Exact Bayesian Inference on Discrete Models via Probability Generating Functions: A Probabilistic Programming Approach

Does a Program Yield the Right Distribution?

Static Posterior Inference of Bayesian Probabilistic Programming via Polynomial Solving

Latticed K-Induction with an Application to Probabilistic Programs

Probabilistic Program Verification Via Inductive Synthesis of Inductive Invariants.

Scaling Exact Inference for Discrete Probabilistic Programs

Beyond the Grounding Bottleneck: Datalog Techniques for Inference in Probabilistic Logic Programs (Technical Report)

Stochastic Probabilistic Programs

Inference Compilation and Universal Probabilistic Programming

A Heavy-Tailed Algebra for Probabilistic Programming

Automated Variational Inference in Probabilistic Programming

Control-Data Separation and Logical Condition Propagation for Efficient Inference on Probabilistic Programs

Declarative Probabilistic Logic Programming in Discrete-Continuous Domains

A Domain Theory for Statistical Probabilistic Programming

Graph polynomials and approximation of partition functions with Loopy Belief Propagation

Explanation-Based Approximate Weighted Model Counting for Probabilistic Logics

Bayesian Optimization for Probabilistic Programs