Abstract:Completely random measures (CRMs) and their normalizations (NCRMs) offer flexible models in Bayesian nonparametrics. But their infinite dimensionality presents challenges for inference. Two popular finite approximations are truncated finite approximations (TFAs) and independent finite approximations (IFAs). While the former have been well-studied, IFAs lack similarly general bounds on approximation error, and there has been no systematic comparison between the two options. In the present work, we propose a general recipe to construct practical finite-dimensional approximations for homogeneous CRMs and NCRMs, in the presence or absence of power laws. We call our construction the automated independent finite approximation (AIFA). Relative to TFAs, we show that AIFAs facilitate more straightforward derivations and use of parallel computing in approximate inference. We upper bound the approximation error of AIFAs for a wide class of common CRMs and NCRMs -- and thereby develop guidelines for choosing the approximation level. Our lower bounds in key cases suggest that our upper bounds are tight. We prove that, for worst-case choices of observation likelihoods, TFAs are more efficient than AIFAs. Conversely, we find that in real-data experiments with standard likelihoods, AIFAs and TFAs perform similarly. Moreover, we demonstrate that AIFAs can be used for hyperparameter estimation even when other potential IFA options struggle or do not apply.

t-divergence Based Approximate Inference

The Logarithmic Super Divergence and its use in Statistical Inference

Greedy Algorithms for Optimal Distribution Approximation

On Bayesian estimation via divergences

Total Variation Distance Meets Probabilistic Inference

Density Approximations for Multivariate Diffusions Via an Ito-Taylor Expansion Approach

Variational Inference for Uncertainty Quantification: an Analysis of Trade-offs

Convergence and asymptotic normality of variational Bayesian approximations for exponential family models with missing values

Probabilistic PCA for T Distributions.

Exact Inference with Approximate Computation for Differentially Private Data via Perturbations

New asymptotic representations of the noncentral $t$-distribution

Trivergence of Probability Distributions, at glance

Independent finite approximations for Bayesian nonparametric inference

Limit Distribution Theory for Quantum Divergences

Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions

Distortion estimates for approximate Bayesian inference

ML ESTIMATION OF THE t DISTRIBUTION USING EM AND ITS EXTENSIONS, ECM AND ECME

A New Delta Expansion for Multivariate Diffusions Via the Itô-Taylor Expansion

Reference and Probability-Matching Priors for the Parameters of a Univariate Student $t$-Distribution

Semi-Nonparametric Estimation of Distribution Divergence in Non-Euclidean Spaces

A new asymptotic representation and inversion method for the Student's t distribution