Abstract:Multivariate, heteroscedastic errors complicate statistical inference in many large-scale denoising problems. Empirical Bayes is attractive in such settings, but standard parametric approaches rest on assumptions about the form of the prior distribution which can be hard to justify and which introduce unnecessary tuning parameters. We extend the nonparametric maximum likelihood estimator (NPMLE) for Gaussian location mixture densities to allow for multivariate, heteroscedastic errors. NPMLEs estimate an arbitrary prior by solving an infinite-dimensional, convex optimization problem; we show that this convex optimization problem can be tractably approximated by a finite-dimensional version. The empirical Bayes posterior means based on an NPMLE have low regret, meaning they closely target the oracle posterior means one would compute with the true prior in hand. We prove an oracle inequality implying that the empirical Bayes estimator performs at nearly the optimal level (up to logarithmic factors) for denoising without prior knowledge. We provide finite-sample bounds on the average Hellinger accuracy of an NPMLE for estimating the marginal densities of the observations. We also demonstrate the adaptive and nearly-optimal properties of NPMLEs for deconvolution. We apply our method to two denoising problems in astronomy, constructing a fully data-driven color-magnitude diagram of 1.4 million stars in the Milky Way and investigating the distribution of 19 chemical abundance ratios for 27 thousand stars in the red clump. We also apply our method to hierarchical linear models, illustrating the advantages of nonparametric shrinkage of regression coefficients on an education data set and on a microarray data set.

High‐dimensional classification based on nonparametric maximum likelihood estimation under unknown and inhomogeneous variances

Robust Estimation and Shrinkage in Ultrahigh Dimensional Expectile Regression with Heavy Tails and Variance Heterogeneity

Semiparametric Expectile Regression for High-dimensional Heavy-tailed and Heterogeneous Data

Expectile regression for analyzing heteroscedasticity in high dimension

An Improved Algorithm for High-Dimensional Continuous Threshold Expectile Model with Variance Heterogeneity

Nonparametric Mean and Variance Adaptive Classification Rule for High-Dimensional Data with Heteroscedastic Variances

A Likelihood Ratio Framework for High Dimensional Semiparametric Regression

Inference for High-Dimensional Linear Expectile Regression with De-Biasing Method

A Nonparametric Bayesian Technique for High-Dimensional Regression

Semiparametric efficient estimation in high‐dimensional partial linear regression models

A Non-Parametric Shrinkage Mean Estimation for Arbitrary Quadratic Loss Functions and Unknown Covariance Matrices

Doubly robust and efficient estimators for heteroscedastic partially linear single-index models allowing high dimensional covariates

Simulation-Based Hypothesis Testing of High Dimensional Means Under Covariance Heterogeneity

High Dimensional Binary Choice Model with Unknown Heteroskedasticity or Instrumental Variables

Variance function estimation in high-dimensions

Mean and variance estimation in high-dimensional heteroscedastic models with non-convex penalties

Estimation of multiple mean vectors in high dimension

Sub-optimality of the Naive Mean Field approximation for proportional high-dimensional Linear Regression

Multivariate, Heteroscedastic Empirical Bayes via Nonparametric Maximum Likelihood

Nonparametric Bayes Classification via Learning of Affine Subspaces

Estimating the mean and variance of a high-dimensional normal distribution using a mixture prior