Abstract:Frequently, population studies feature pyramidally-organized data represented using Hierarchical Bayesian Models (HBM) enriched with plates. These models can become prohibitively large in settings such as neuroimaging, where a sample is composed of a functional MRI signal measured on 300 brain locations, across 4 measurement sessions, and 30 subjects, resulting in around 1 million latent <a class="link-external link-http" href="http://parameters.Such" rel="external noopener nofollow">this http URL</a> high dimensionality hampers the usage of modern, expressive flow-based <a class="link-external link-http" href="http://techniques.To" rel="external noopener nofollow">this http URL</a> infer parameter posterior distributions in this challenging class of problems, we designed a novel methodology that automatically produces a variational family dual to a target HBM. This variational family, represented as a neural network, consists in the combination of an attention-based hierarchical encoder feeding summary statistics to a set of normalizing flows. Our automatically-derived neural network exploits exchangeability in the plate-enriched HBM and factorizes its parameter space. The resulting architecture reduces by orders of magnitude its parameterization with respect to that of a typical flow-based representation, while maintaining <a class="link-external link-http" href="http://expressivity.Our" rel="external noopener nofollow">this http URL</a> method performs inference on the specified HBM in an amortized setup: once trained, it can readily be applied to a new data sample to compute the parameters' full <a class="link-external link-http" href="http://posterior.We" rel="external noopener nofollow">this http URL</a> demonstrate the capability and scalability of our method on simulated data, as well as a challenging high-dimensional brain parcellation experiment. We also open up several questions that lie at the intersection between normalizing flows, SBI, structured Variational Inference, and inference amortization.

Automatic structured variational inference

Automatic Variational Inference in Stan

Automatic variational inference with cascading flows

Automated Variational Inference in Probabilistic Programming

Variational Inference: A Review for Statisticians

Provably Scalable Black-Box Variational Inference with Structured Variational Families

Accelerated Stochastic Variational Inference

PAVI: Plate-Amortized Variational Inference

Rethinking Variational Inference for Probabilistic Programs with Stochastic Support

Variational Autoencoders for Efficient Simulation-Based Inference

Variational Message Passing with Structured Inference Networks

Probabilistic Programming with Programmable Variational Inference

Amortized Variational Inference: When and Why?

Structured Black Box Variational Inference for Latent Time Series Models

ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models

Variational Gibbs Inference for Statistical Model Estimation from Incomplete Data

Variational inference: uncertainty quantification in additive models

On the Convergence of Coordinate Ascent Variational Inference

Wasserstein Gradient Flow over Variational Parameter Space for Variational Inference

Improving Variational Autoencoders with Inverse Autoregressive Flow

Two Methods For Wild Variational Inference