Abstract:It contains the proofs of the main results in this paper and additional simulation results. In this work, we propose a scalable Bayesian procedure for learning the local dependence structure in a high-dimensional model where the variables possess a natural ordering. The ordering of variables can be indexed by time, the vicinities of spatial locations, and so on, with the natural assumption that variables far apart tend to have weak correlations. Applications of such models abound in a variety of fields such as finance, genome associations analysis and spatial modeling. We adopt a flexible framework under which each variable is dependent on its neighbors or predecessors, and the neighborhood size can vary for each variable. It is of great interest to reveal this local dependence structure by estimating the covariance or precision matrix while yielding a consistent estimate of the varying neighborhood size for each variable. The existing literature on banded covariance matrix estimation, which assumes a fixed bandwidth cannot be adapted for this general setup. We employ the modified Cholesky decomposition for the precision matrix and design a flexible prior for this model through appropriate priors on the neighborhood sizes and Cholesky factors. The posterior contraction rates of the Cholesky factor are derived which are nearly or exactly minimax optimal, and our procedure leads to consistent estimates of the neighborhood size for all the variables. Another appealing feature of our procedure is its scalability to models with large numbers of variables due to efficient posterior inference without resorting to MCMC algorithms. Numerical comparisons are carried out with competitive methods, and applications are considered for some real datasets.

Provably Scalable Black-Box Variational Inference with Structured Variational Families

Structured Black Box Variational Inference for Latent Time Series Models

On the Convergence of Black-Box Variational Inference

Automatic structured variational inference

Variational Inference for Large Bayesian Vector Autoregressions

Provable convergence guarantees for black-box variational inference

Practical and Matching Gradient Variance Bounds for Black-Box Variational Bayesian Inference

A Framework for Improving the Reliability of Black-box Variational Inference

Scalable Variational Inference in Log-supermodular Models

Scalable Random Feature Latent Variable Models

Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection

Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertainty

Scalable Bayesian High-dimensional Local Dependence Learning

Batch and match: black-box variational inference with a score-based divergence

A Measure-Theoretic Variational Bayesian Algorithm for Large Dimensional Problems

Variance Control for Black Box Variational Inference Using The James-Stein Estimator

On the Convergence of Coordinate Ascent Variational Inference

PAVI: Plate-Amortized Variational Inference

Variational inference for large-scale models of discrete choice

Structured Optimal Variational Inference for Dynamic Latent Space Models

Sparse Bayesian Neural Networks: Bridging Model and Parameter Uncertainty through Scalable Variational Inference