Abstract:Multivariate, heteroscedastic errors complicate statistical inference in many large-scale denoising problems. Empirical Bayes is attractive in such settings, but standard parametric approaches rest on assumptions about the form of the prior distribution which can be hard to justify and which introduce unnecessary tuning parameters. We extend the nonparametric maximum likelihood estimator (NPMLE) for Gaussian location mixture densities to allow for multivariate, heteroscedastic errors. NPMLEs estimate an arbitrary prior by solving an infinite-dimensional, convex optimization problem; we show that this convex optimization problem can be tractably approximated by a finite-dimensional version. The empirical Bayes posterior means based on an NPMLE have low regret, meaning they closely target the oracle posterior means one would compute with the true prior in hand. We prove an oracle inequality implying that the empirical Bayes estimator performs at nearly the optimal level (up to logarithmic factors) for denoising without prior knowledge. We provide finite-sample bounds on the average Hellinger accuracy of an NPMLE for estimating the marginal densities of the observations. We also demonstrate the adaptive and nearly-optimal properties of NPMLEs for deconvolution. We apply our method to two denoising problems in astronomy, constructing a fully data-driven color-magnitude diagram of 1.4 million stars in the Milky Way and investigating the distribution of 19 chemical abundance ratios for 27 thousand stars in the red clump. We also apply our method to hierarchical linear models, illustrating the advantages of nonparametric shrinkage of regression coefficients on an education data set and on a microarray data set.

Bayes and empirical Bayes: do they merge?

Empirical Bayes in Bayesian learning: understanding a common practice

Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem

Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference

Matching Bayesian and frequentist coverage probabilities when using an approximate data covariance matrix

Bayesian data fusion with shared priors

Multivariate, Heteroscedastic Empirical Bayes via Nonparametric Maximum Likelihood

Empirical Bayes factors for common hypothesis tests

Empirical partially Bayes multiple testing and compound $χ^2$ decisions

Learning from a lot: Empirical Bayes in high-dimensional prediction settings

Empirical Bayes Methods for Prior Estimation in Systems Medicine

Empirical Bayes for Large-scale Randomized Experiments: a Spectral Approach

Joining Forces of Bayesian and Frequentist Methodology: A Study for Inference in the Presence of Non-Identifiability

Merging Rate of Opinions via Optimal Transport on Random Measures

Resolving conflicts between statistical methods by probability combination: Application to empirical Bayes analyses of genomic data

Empirical Bayes via ERM and Rademacher complexities: the Poisson model

Why Dempster’s Rule Doesn’t Behave As Bayes Rule with Informative Priors

Convergence of Pseudo-Bayes Factors in Forward and Inverse Regression Problems

Empirical Bayes Methods, Evidentialism, and the Inferential Roles They Play

Empirical Bayes When Estimation Precision Predicts Parameters

Assessing replication success via skeptical mixture priors