Abstract:In a regular full exponential family, the maximum likelihood estimator (MLE) need not exist in the traditional sense. However, the MLE may exist in the completion of the exponential family. Existing algorithms for finding the MLE in the completion solve many linear programs; they are slow in small problems and too slow for large problems. We provide new, fast, and scalable methodology for finding the MLE in the completion of the exponential family. This methodology is based on conventional maximum likelihood computations which come close, in a sense, to finding the MLE in the completion of the exponential family. These conventional computations construct a likelihood maximizing sequence of canonical parameter values which goes uphill on the likelihood function until they meet a convergence criteria. Nonexistence of the MLE in this context results from a degeneracy of the canonical statistic of the exponential family, the canonical statistic is on the boundary of its support. There is a correspondance between this boundary and the null eigenvectors of the Fisher information matrix. Convergence of Fisher information along a likelihood maximizing sequence follows from cumulant generating function (CGF) convergence along a likelihood maximizing sequence, conditions for which are given. This allows for the construction of necessarily one-sided confidence intervals for mean value parameters when the MLE exists in the completion. We demonstrate our methodology on three examples in the main text and three additional examples in the Appendix. We show that when the MLE exists in the completion of the exponential family, our methodology provides statistical inference that is much faster than existing techniques.

Using Maximum Weighted Likelihood to Derive Lehmer and Hölder Families of Means

Using maximum weighted likelihood to derive Lehmer and Hölder mean families

Deriving Lehmer and Hölder means as maximum weighted likelihood estimates for the multivariate exponential family

Generalized weight approximation by maximal families with applications

On the maximum likelihood estimation in general log-linear models

On Weight Design of Maximum Weighted Likelihood and an Extended EM Algorithm

On variational inference and maximum likelihood estimation with the λ-exponential family

Bias-Corrected Maximum Likelihood Estimation of the Parameters of the Weighted Lindley Distribution

Maximum Likelihood from Incomplete Data Via the EM Algorithm

Computationally efficient likelihood inference in exponential families when the maximum likelihood estimator does not exist

Penalized Maximum Likelihood Estimator for Mixture of von Mises-Fisher Distributions

Interacting Particle Langevin Algorithm for Maximum Marginal Likelihood Estimation

Nonparametric maximum likelihood estimation under a likelihood ratio order

Essential formulae for restricted maximum likelihood and its derivatives associated with the linear mixed models

Tilting maximum Lq-Likelihood estimation for extreme values drawing on block maxima

On the Maximum Likelihood Estimation of Population and Domain Means

On the MLE of the Waring Distribution

Conditional Density Estimation via Weighted Logistic Regressions

Maximum Probability and Relative Entropy Maximization. Bayesian Maximum Probability and Empirical Likelihood

Maximum Entropy Estimator for Hidden Markov Models: Reduction to Dimension 2

A family of discrete maximum-entropy distributions