Abstract:Frequently, population studies feature pyramidally-organized data represented using Hierarchical Bayesian Models (HBM) enriched with plates. These models can become prohibitively large in settings such as neuroimaging, where a sample is composed of a functional MRI signal measured on 300 brain locations, across 4 measurement sessions, and 30 subjects, resulting in around 1 million latent <a class="link-external link-http" href="http://parameters.Such" rel="external noopener nofollow">this http URL</a> high dimensionality hampers the usage of modern, expressive flow-based <a class="link-external link-http" href="http://techniques.To" rel="external noopener nofollow">this http URL</a> infer parameter posterior distributions in this challenging class of problems, we designed a novel methodology that automatically produces a variational family dual to a target HBM. This variational family, represented as a neural network, consists in the combination of an attention-based hierarchical encoder feeding summary statistics to a set of normalizing flows. Our automatically-derived neural network exploits exchangeability in the plate-enriched HBM and factorizes its parameter space. The resulting architecture reduces by orders of magnitude its parameterization with respect to that of a typical flow-based representation, while maintaining <a class="link-external link-http" href="http://expressivity.Our" rel="external noopener nofollow">this http URL</a> method performs inference on the specified HBM in an amortized setup: once trained, it can readily be applied to a new data sample to compute the parameters' full <a class="link-external link-http" href="http://posterior.We" rel="external noopener nofollow">this http URL</a> demonstrate the capability and scalability of our method on simulated data, as well as a challenging high-dimensional brain parcellation experiment. We also open up several questions that lie at the intersection between normalizing flows, SBI, structured Variational Inference, and inference amortization.

Online Variational Bayes Inference for High-Dimensional Correlated Data

Variational Inference for Large Bayesian Vector Autoregressions

ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models

Ultra-Fast Approximate Inference Using Variational Functional Mixed Models

Variational Bayesian Inference in High-Dimensional Linear Mixed Models

Covariance Regression with High-Dimensional Predictors

A Nonparametric Bayesian Technique for High-Dimensional Regression

Variational Learning Data Fusion with Unknown Correlation

Scalable Bayesian regression in high dimensions with multiple data sources

A Variational Approach for Modeling High-dimensional Spatial Generalized Linear Mixed Models

Variational Bayesian hierarchical regression for data analysis

A variational Bayes approach to debiased inference for low-dimensional parameters in high-dimensional linear regression

High-dimensional copula variational approximation through transformation

Distributed Variational Bayesian Algorithms Over Sensor Networks

Distributed Variational Inference for Online Supervised Learning

Doubly Distributed Supervised Learning and Inference with High-Dimensional Correlated Outcomes

Fast approximate inference for multivariate longitudinal data

Variational Bayesian partially linear mean shift models for high-dimensional Alzheimer's disease neuroimaging data

Variational Approximation of Factor Stochastic Volatility Models

Multi-Channel Stochastic Variational Inference for the Joint Analysis of Heterogeneous Biomedical Data in Alzheimer's Disease

Scalable Modeling of Spatiotemporal Data using the Variational Autoencoder: an Application in Glaucoma