Abstract:Computing the marginal likelihood (also called the Bayesian model evidence) is an important task in Bayesian model selection, providing a principled quantitative way to compare models. The learned harmonic mean estimator solves the exploding variance problem of the original harmonic mean estimation of the marginal likelihood. The learned harmonic mean estimator learns an importance sampling target distribution that approximates the optimal distribution. While the approximation need not be highly accurate, it is critical that the probability mass of the learned distribution is contained within the posterior in order to avoid the exploding variance problem. In previous work a bespoke optimization problem is introduced when training models in order to ensure this property is satisfied. In the current article we introduce the use of normalizing flows to represent the importance sampling target distribution. A flow-based model is trained on samples from the posterior by maximum likelihood estimation. Then, the probability density of the flow is concentrated by lowering the variance of the base distribution, i.e. by lowering its "temperature", ensuring its probability mass is contained within the posterior. This approach avoids the need for a bespoke optimisation problem and careful fine tuning of parameters, resulting in a more robust method. Moreover, the use of normalizing flows has the potential to scale to high dimensional settings. We present preliminary experiments demonstrating the effectiveness of the use of flows for the learned harmonic mean estimator. The harmonic code implementing the learned harmonic mean, which is publicly available, has been updated to now support normalizing flows.

Approximation of Intractable Likelihood Functions in Systems Biology via Normalizing Flows

Efficient CDF Approximations for Normalizing Flows

Verlet Flows: Exact-Likelihood Integrators for Flow-Based Generative Models

Flows for Flows: Training Normalizing Flows Between Arbitrary Distributions with Maximum Likelihood Estimation

Flow Matching for Scalable Simulation-Based Inference

Stochastic Normalizing Flows

Piecewise Normalizing Flows

Learned harmonic mean estimation of the marginal likelihood with normalizing flows

A Tale of Two Latent Flows: Learning Latent Space Normalizing Flow with Short-run Langevin Flow for Approximate Inference

Normalizing Flows for Bayesian Posteriors: Reproducibility and Deployment

Latent Variable Modelling with Hyperbolic Normalizing Flows

Combining Normalizing Flows and Quasi-Monte Carlo

BayesFlow: Learning complex stochastic models with invertible neural networks

Improved Variational Bayesian Phylogenetic Inference with Normalizing Flows

Functional normalizing flow for statistical inverse problems of partial differential equations

The Expressive Power of a Class of Normalizing Flow Models

Uncertainty quantification and out-of-distribution detection using surjective normalizing flows

Arbitrary Distributions Mapping via SyMOT-Flow: A Flow-based Approach Integrating Maximum Mean Discrepancy and Optimal Transport

Conditional Pseudo-Reversible Normalizing Flow for Surrogate Modeling in Quantifying Uncertainty Propagation

Preferential Normalizing Flows

Normalizing Flows for Microscopic Many-Body Calculations: An Application to the Nuclear Equation of State