Abstract:We employ a parameter-free distribution estimation framework where estimators are random distributions and utilize the Kullback-Leibler (KL) divergence as a loss function. Wu and Vos [J. Statist. Plann. Inference 142 (2012) 1525-1536] show that when an estimator obtained from an i.i.d. sample is viewed as a random distribution, the KL risk of the estimator decomposes in a fashion parallel to the mean squared error decomposition when the estimator is a real-valued random variable. In this paper, we explore how conditional versions of distribution expectation ($E^{\dagger}$) can be defined so that a distribution version of the Rao-Blackwell theorem holds. We define distributional expectation and variance ($V^{\dagger}$) that also provide a decomposition of KL risk in exponential and mixture families. For exponential families, we show that the maximum likelihood estimator (viewed as a random distribution) is distribution unbiased and is the unique uniformly minimum distribution variance unbiased (UMV$^{\dagger}$U) estimator. Furthermore, we show that the MLE is robust against model specification in that if the true distribution does not belong to the exponential family, the MLE is UMV$^{\dagger}$U for the KL projection of the true distribution onto the exponential families provided these two distribution have the same expectation for the canonical statistic. To allow for estimators taking values outside of the exponential family, we include results for KL projection and define an extended projection to accommodate the non-existence of the MLE for families having discrete sample space. Illustrative examples are provided.

On an Asymptotic Distribution for the MLE

Asymptotic expansion for nonparametric M-estimator in a nonlinear regression model with long-memory errors

Asymptotic Distributions of Innovation Density Estimators in Linear Processes

On the MLE of the Waring Distribution

Asymptotic Properties of Nonparametric M-Estimation for Mixing Functional Data

On the Asymptotic Approximation of Inverse Moment under Sub-Linear Expectations

Asymptotics for minimisers of convex processes

Asymptotic Theory of Least Distances Estimate in Multivariate Linear Models

A Note on Parameter Asymptotics for Weighted Lindley Distribution

On the Symmetry of Limiting Distributions of M-estimators

Self-Normalized Moderate Deviation and Laws of the Iterated Logarithm under G-Expectation

ASYMPTOTICAL EFFICIENCY OFMLE WITH GROUPED DATA

Bias-Corrected Maximum Likelihood Estimation of the Parameters of the Weighted Lindley Distribution

On the Approximate Maximum Likelihood Estimation for Diffusion Processes

The MLE is minimax optimal for LGC

Asymptotic Efficiency of Randomly Weighted Bootstrap for Linear Models

Maximum likelihood estimators uniformly minimize distribution variance among distribution unbiased estimators in exponential families

Deterministic Inequalities for Smooth M-estimators

Asymptotic Normality of Maximum Quasi-Likelihood Estimators in Generalized Linear Models with Fixed Design

Weak consistency and asymptotic normality of quasi-maximum likelihood estimates in generalized linear models

Asymptotic distribution for two-sided tests with lower and upper boundaries on the parameter of interest