Abstract:In this paper, we are interested in Bayesian inverse problems where either the data fidelity term or the prior distribution is Gaussian or driven from a hierarchical Gaussian model. Generally, Markov chain Monte Carlo (MCMC) algorithms allow us to generate sets of samples that are employed to infer some relevant parameters of the underlying distributions. However, when the parameter space is high-dimensional, the performance of stochastic sampling algorithms is very sensitive to existing dependencies between parameters. In particular, this problem arises when one aims to sample from a high-dimensional Gaussian distribution whose covariance matrix does not present a simple structure. Another challenge is the design of Metropolis-Hastings proposals that make use of information about the local geometry of the target density in order to speed up the convergence and improve mixing properties in the parameter space, while not being too computationally expensive. These two contexts are mainly related to the presence of two heterogeneous sources of dependencies stemming either from the prior or the likelihood in the sense that the related covariance matrices cannot be diagonalized in the same basis. In this work, we address these two issues. Our contribution consists of adding auxiliary variables to the model in order to dissociate the two sources of dependencies. In the new augmented space, only one source of correlation remains directly related to the target parameters, the other sources of correlations being captured by the auxiliary variables. Experiments are conducted on two practical image restoration problems-namely the recovery of multichannel blurred images embedded in Gaussian noise and the recovery of signal corrupted by a mixed Gaussian noise. Experimental results indicate that adding the proposed auxiliary variables makes the sampling problem simpler since the new conditional distribution no longer contains highly heterogeneous correlations. Thus, the computational cost of each iteration of the Gibbs sampler is significantly reduced while ensuring good mixing properties.

Scalability of Metropolis-within-Gibbs schemes for high-dimensional Bayesian models

Auxiliary MCMC and particle Gibbs samplers for parallelisable inference in latent dynamical systems

Optimal Scaling of Random-Walk Metropolis Algorithms on General Target Distributions

Sampling hyperparameters in hierarchical models: improving on Gibbs for high-dimensional latent fields and large data sets

Convergence of Dirichlet Forms for MCMC Optimal Scaling with Dependent Target Distributions on Large Graphs

A Vine-Copula Based Adaptive Mcmc Sampler for Efficient Inference of Dynamical Systems

Robust random walk-like Metropolis-Hastings algorithms for concentrating posteriors

Gibbs sampling for a Bayesian hierarchical general linear model

Scalable Stochastic Gradient Riemannian Langevin Dynamics in Non-Diagonal Metrics

MCMC-Based Inference in the Era of Big Data: A Fundamental Analysis of the Convergence Complexity of High-Dimensional Chains

Accelerating Markov Chain Monte Carlo sampling with diffusion models

An Auxiliary Variable Method for Markov Chain Monte Carlo Algorithms in High Dimension

Multigrid Monte Carlo Revisited: Theory and Bayesian Inference

Conjugate gradient methods for high-dimensional GLMMs

Asymptotic bias of inexact Markov Chain Monte Carlo methods in high dimension

Statistical guarantees for stochastic Metropolis-Hastings

Adapting The Gibbs Sampler

Scalable Bayesian inference for the generalized linear mixed model

Improving sampling by modifying the effective diffusion

Entropy contraction of the Gibbs sampler under log-concavity

Multilevel Markov Chain Monte Carlo with likelihood scaling for high-resolution data assimilation