Abstract:In applications of Bayesian procedures, even when the prior law is carefully specified, it may be delicate to elicit the prior hyperparameters so that it is often tempting to fix them from the data, usually by their maximum likelihood estimates (MMLE), obtaining a so-called empirical Bayes posterior distribution. Although questionable, this is a common practice; but theoretical properties seem mostly only available on a case-by-case basis. In this paper we provide general properties for parametric models. First, we study the limit behavior of the MMLE and prove results in quite general settings, while also conceptualizing the frequentist context as an unexplored case of maximum likelihood estimation under model misspecification. We cover both identifiable models, illustrating applications to sparse regression, and non-identifiable models - specifically, overfitted mixture models. Finally, we prove higher order merging results. In regular cases, the empirical Bayes posterior is shown to be a fast approximation to the Bayesian posterior distribution of the researcher who, within the given class of priors, has the most information about the true model's parameters. This is a faster approximation than classic Bernstein-von Mises results. Given the class of priors, our work provides formal contents to common beliefs on this popular practice.

Empirical Bayes estimates of steady-state availability in exponential distribution model

Parameter Estimation in Exponential Models by Linear and Nonlinear Fitting Methods.

Probability Method for Foundation Settlement Simulated with Poisson Curve

Parameter Estimation for Settlement Prediction Model Using Bayesian Inference Approach

Empirical Bayes in Bayesian learning: understanding a common practice

A Note on the Convergence Rates for Empirical Bayes Estimators of Parameters in Multiple-Parameter Exponential Families

A Bayesian View on the Polynomial Distribution Model in Estimation of Distribution Algorithms

The empirical Bayes estimators of the rate parameter of the gamma distribution with a conjugate gamma prior under Stein's loss function

Empirical Bayes Test for Two-parameter Exponential Distribution under Type-II Censored Samples

On Empirical Bayes Estimation of Variance Components in Random Effects Model

A Bayesian Analysis for the Parameters of the Exponential-Logarithmic Distribution

Parameter estimation in exponential failure rate model using masked data

A General Framework for Empirical Bayes Estimation in Discrete Linear Exponential Family

Bayesian estimation of the number of species from Poisson-Lindley stochastic abundance model using non-informative priors

Empirical Bayes Methods for Prior Estimation in Systems Medicine

Empirical Bayes via ERM and Rademacher complexities: the Poisson model

EMPIRICAL BAYESIAN ESTIMATION OF THE LOCATION PARAMETER IN DOUBLE EXPONENTIAL DISTRIBUTION

Empirical Bayes estimation for the stochastic blockmodel

ASYMPTOTICALLY OPTIMAL EMPIRICAL BAYES ESTIMATION FOR THE PARAMETERS OF MULTI-PARAMETER DISCRETE EXPONENTIAL FAMILY

The Robustness of Bayes Linear Unbiased Estimations under Misspecified Prior Assumption

The Empirical Bayes Estimation Problem for Continuous One-parameter Exponential Family:In the case of NA Samples