Abstract:We derive lower bounds on the Bayes risk in decentralized estimation, where the estimator does not have direct access to the random samples generated conditionally on the random parameter of interest, but only to the data received from local processors that observe the samples. The received data are subject to communication constraints, due to quantization and the noise in the communication channels from the processors to the estimator. We first derive general lower bounds on the Bayes risk using information-theoretic quantities, such as mutual information, information density, small ball probability, and differential entropy. We then apply these lower bounds to the decentralized case, using strong data processing inequalities to quantify the contraction of information due to communication constraints. We treat the cases of a single processor and of multiple processors, where the samples observed by different processors may be conditionally dependent given the parameter, for noninteractive and interactive communication protocols. Our results recover and improve recent lower bounds on the Bayes risk and the minimax risk for certain decentralized estimation problems, where previously only conditionally independent sample sets and noiseless channels have been considered. Moreover, our results provide a general way to quantify the degradation of estimation performance caused by distributing resources to multiple processors, which is only discussed for specific examples in existing works.

On Bayes risk of the posterior mean in linear inverse problems

A posterior contraction for Bayesian inverse problems in Banach spaces

Non-asymptotic error estimates for the Laplace approximation in Bayesian inverse problems

On the Bernstein-Von Mises Theorem for High Dimensional Nonlinear Bayesian Inverse Problems

On the Frequentist Coverage of Bayes Posteriors in Nonlinear Inverse Problems

Generalized Bayes Approach to Inverse Problems with Model Misspecification

Optimal low-rank approximations of posteriors for linear Gaussian inverse problems on Hilbert spaces

On the robustness of posterior means

On the convergence of the Laplace approximation and noise-level-robustness of Laplace-based Monte Carlo methods for Bayesian inverse problems

Oracle-type Posterior Contraction Rates in Bayesian Inverse Problems

Variational Gaussian Processes For Linear Inverse Problems

Semi-parametric Bernstein-von Mises in Linear Inverse Problems

Mixed Laplace approximation for marginal posterior and Bayesian inference in error-in-operator model

A Bayesian approach with Gaussian priors to the inverse problem of source identification in elliptic PDEs

Are minimizers of the Onsager-Machlup functional strong posterior modes?

Random forward models and log-likelihoods in Bayesian inverse problems

Posterior contraction rates in a sparse non-linear mixed-effects model

Statistical guarantees for Bayesian uncertainty quantification in non-linear inverse problems with Gaussian process priors

Posterior distribution existence and error control in Banach spaces in the Bayesian approach to UQ in inverse problems

Information-Theoretic Lower Bounds on Bayes Risk in Decentralized Estimation

On some information-theoretic aspects of non-linear statistical inverse problems