Abstract:Multivariate, heteroscedastic errors complicate statistical inference in many large-scale denoising problems. Empirical Bayes is attractive in such settings, but standard parametric approaches rest on assumptions about the form of the prior distribution which can be hard to justify and which introduce unnecessary tuning parameters. We extend the nonparametric maximum likelihood estimator (NPMLE) for Gaussian location mixture densities to allow for multivariate, heteroscedastic errors. NPMLEs estimate an arbitrary prior by solving an infinite-dimensional, convex optimization problem; we show that this convex optimization problem can be tractably approximated by a finite-dimensional version. The empirical Bayes posterior means based on an NPMLE have low regret, meaning they closely target the oracle posterior means one would compute with the true prior in hand. We prove an oracle inequality implying that the empirical Bayes estimator performs at nearly the optimal level (up to logarithmic factors) for denoising without prior knowledge. We provide finite-sample bounds on the average Hellinger accuracy of an NPMLE for estimating the marginal densities of the observations. We also demonstrate the adaptive and nearly-optimal properties of NPMLEs for deconvolution. We apply our method to two denoising problems in astronomy, constructing a fully data-driven color-magnitude diagram of 1.4 million stars in the Milky Way and investigating the distribution of 19 chemical abundance ratios for 27 thousand stars in the red clump. We also apply our method to hierarchical linear models, illustrating the advantages of nonparametric shrinkage of regression coefficients on an education data set and on a microarray data set.

MassBayes: A New Generative Classifier with Multi-dimensional Likelihood Estimation.

Max-margin Basedbayesian Classifier

A Generic Ensemble Approach to Estimate Multidimensional Likelihood in Bayesian Classifier Learning.

Multi-dimensional Mass Estimation and Mass-based Clustering

Learning by Extrapolation from Marginal to Full-Multivariate Probability Distributions: Decreasingly Naive Bayesian Classification

Nonparametric Bayesian Multi-task Learning with Max-margin Posterior Regularization

PQMass: Probabilistic Assessment of the Quality of Generative Models using Probability Mass Estimation

Nonparametric Bayes Classification via Learning of Affine Subspaces

Efficient heuristics for learning scalable Bayesian network classifier from labeled and unlabeled data

LCBM: A Multi-View Probabilistic Model for Multi-Label Classification

Efficient Bayesian High-Dimensional Classification via Random Projection with Application to Gene Expression Data

Probability-based Label Enhancement for Multi-Dimensional Classification

Generative Maximum Entropy Learning for Multiclass Classification

The Bridged Posterior: Optimization, Profile Likelihood and a New Approach to Generalized Bayes

A Double Layer Bayesian Classifier Using Conditional Mutual Information

Bayesian Conditional Gaussian Network Classifiers with Applications to Mass Spectra Classification

Bayesian Max-margin Multi-Task Learning with Data Augmentation.

Generative AI for Bayesian Computation

Multivariate, Heteroscedastic Empirical Bayes via Nonparametric Maximum Likelihood

Revisiting Discriminative Vs. Generative Classifiers: Theory and Implications

Nonparametric Bayesian Multi-Task Large-margin Classification.