Uniqueness of the maximum likelihood estimator for k-monotone densities

Arseni Seregin

DOI: https://doi.org/10.48550/arXiv.1001.1782

2010-01-12

Abstract:We prove uniqueness of the maximum likelihood estimator for the class of k-monotone densities.

Statistics Theory

What problem does this paper attempt to address?

Limit theorems for kernel density estimators under dependent samples

yuexu zhao,zhengyan lin

2013-01-01

Abstract: In this paper, we construct a moment inequality for mixing dependent random variables, it is of independent interest. As applications, the consistency of the kernel density estimation is investigated. Several limit theorems are established: First, the central limit theorems for the kernel density estimator $f_{n,K}(x)$ and its distribution function are constructed. Also, the convergence rates of $\|f_{n,K}(x)-Ef_{n,K}(x)\|_{p}$ in sup-norm loss and integral $L^{p}$-norm loss are proved. Moreover, the a.s. convergence rates of the supremum of $|f_{n,K}(x)-Ef_{n,K}(x)|$ over a compact set and the whole real line are obtained. It is showed, under suitable conditions on the mixing rates, the kernel function and the bandwidths, that the optimal rates for i.i.d. random variables are also optimal for dependent ones.
On the Maximum Likelihood Estimators' Uniqueness and Existence for Two Unitary Distributions: Analytically and Graphically, with Application

Gadir Alomair,Yunus Akdoğan,Hassan S. Bakouch,Tenzile Erbayram

DOI: https://doi.org/10.3390/sym16050610

2024-05-15

Symmetry

Abstract:Unit distributions, exhibiting inherent symmetrical properties, have been extensively studied across various fields. A significant challenge in these studies, particularly evident in parameter estimations, is the existence and uniqueness of estimators. Often, it is challenging to demonstrate the existence of a unique estimator. The major issue with maximum likelihood and other estimator-finding methods that use iterative methods is that they need an initial value to reach the solution. This dependency on initial values can lead to local extremes that fail to represent the global extremities, highlighting a lack of symmetry in solution robustness. This study applies a very simple, and unique, estimation method for unit Weibull and unit Burr XII distributions that both attain the global maximum value. Therefore, we can conclude that the findings from the obtained propositions demonstrate that both the maximum likelihood and graphical methods are symmetrically similar. In addition, three real-world data applications are made to show that the method works efficiently.

multidisciplinary sciences
Bayesian Inference for $k$-Monotone Densities with Applications to Multiple Testing

Kang Wang,Subhashis Ghosal

2023-06-08

Abstract:Shape restriction, like monotonicity or convexity, imposed on a function of interest, such as a regression or density function, allows for its estimation without smoothness assumptions. The concept of $k$-monotonicity encompasses a family of shape restrictions, including decreasing and convex decreasing as special cases corresponding to $k=1$ and $k=2$. We consider Bayesian approaches to estimate a $k$-monotone density. By utilizing a kernel mixture representation and putting a Dirichlet process or a finite mixture prior on the mixing distribution, we show that the posterior contraction rate in the Hellinger distance is $(n/\log n)^{- k/(2k + 1)}$ for a $k$-monotone density, which is minimax optimal up to a polylogarithmic factor. When the true $k$-monotone density is a finite $J_0$-component mixture of the kernel, the contraction rate improves to the nearly parametric rate $\sqrt{(J_0 \log n)/n}$. Moreover, by putting a prior on $k$, we show that the same rates hold even when the best value of $k$ is unknown. A specific application in modeling the density of $p$-values in a large-scale multiple testing problem is considered. Simulation studies are conducted to evaluate the performance of the proposed method.

Statistics Theory,Methodology
A UNIFIED STUDY OF NONPARAMETRIC INFERENCE FOR MONOTONE FUNCTIONS

Ted Westling,Marco Carone

DOI: https://doi.org/10.1214/19-aos1835

Abstract:The problem of nonparametric inference on a monotone function has been extensively studied in many particular cases. Estimators considered have often been of so-called Grenander type, being representable as the left derivative of the greatest convex minorant or least concave majorant of an estimator of a primitive function. In this paper, we provide general conditions for consistency and pointwise convergence in distribution of a class of generalized Grenander-type estimators of a monotone function. This broad class allows the minorization or majoratization operation to be performed on a data-dependent transformation of the domain, possibly yielding benefits in practice. Additionally, we provide simpler conditions and more concrete distributional theory in the important case that the primitive estimator and data-dependent transformation function are asymptotically linear. We use our general results in the context of various well-studied problems, and show that we readily recover classical results established separately in each case. More importantly, we show that our results allow us to tackle more challenging problems involving parameters for which the use of flexible learning strategies appears necessary. In particular, we study inference on monotone density and hazard functions using informatively right-censored data, extending the classical work on independent censoring, and on a covariate-marginalized conditional mean function, extending the classical work on monotone regression functions.
On uniqueness of the set of k-means

Javier Cárcamo,Antonio Cuevas,Luis A. Rodríguez

2024-10-17

Abstract:We provide necessary and sufficient conditions for the uniqueness of the k-means set of a probability distribution. This uniqueness problem is related to the choice of k: depending on the underlying distribution, some values of this parameter could lead to multiple sets of k-means, which hampers the interpretation of the results and/or the stability of the algorithms. We give a general assessment on consistency of the empirical k-means adapted to the setting of non-uniqueness and determine the asymptotic distribution of the within cluster sum of squares (WCSS). We also provide statistical characterizations of k-means uniqueness in terms of the asymptotic behavior of the empirical WCSS. As a consequence, we derive a bootstrap test for uniqueness of the set of k-means. The results are illustrated with examples of different types of non-uniqueness and we check by simulations the performance of the proposed methodology.

Statistics Theory,Methodology,Machine Learning
Bayesian Inference for Multivariate Monotone Densities

Kang Wang,Subhashis Ghosal

2023-06-08

Abstract:We consider a nonparametric Bayesian approach to estimation and testing for a multivariate monotone density. Instead of following the conventional Bayesian route of putting a prior distribution complying with the monotonicity restriction, we put a prior on the step heights through binning and a Dirichlet distribution. An arbitrary piece-wise constant probability density is converted to a monotone one by a projection map, taking its $\mathbb{L}_1$-projection onto the space of monotone functions, which is subsequently normalized to integrate to one. We construct consistent Bayesian tests to test multivariate monotonicity of a probability density based on the $\mathbb{L}_1$-distance to the class of monotone functions. The test is shown to have a size going to zero and high power against alternatives sufficiently separated from the null hypothesis. To obtain a Bayesian credible interval for the value of the density function at an interior point with guaranteed asymptotic frequentist coverage, we consider a posterior quantile interval of an induced map transforming the function value to its value optimized over certain blocks. The limiting coverage is explicitly calculated and is seen to be higher than the credibility level used in the construction. By exploring the asymptotic relationship between the coverage and the credibility, we show that a desired asymptomatic coverage can be obtained exactly by starting with an appropriate credibility level.

Statistics Theory,Methodology
A note on the convergence rate of the kernel density estimator of the mode

Xiaoping Shi,Yuehua Wu,Baiqi Miao

DOI: https://doi.org/10.1016/j.spl.2009.05.015

IF: 0.718

2009-01-01

Statistics & Probability Letters

Abstract:In this paper, the mode estimator based on the Parzen–Rosenblatt kernel estimator is considered [Parzen, E., 1962. On estimating probability density function and mode. The Annals of Mathematical Statistics 33, 1065–1076]. In light of Shi et al. [Shi, X., Wu, Y., Miao, B., 2009. Strong convergence rate of estimators of change point and its application. Computational Statistics & Data Analysis 53, 990–998], under mild conditions, we establish the relationship between the convergence rate of the mode estimator and the window width. In this way, we obtain a better convergence rate of the mode estimator.
On the tails of log-concave density estimators

Didier B. Ryter,Lutz Duembgen

2024-09-26

Abstract:This note proves that the nonparametric maximum likelihood estimator of a univariate log-concave probability density satisfies some consistency properties in the tail regions.

Statistics Theory
Maximum Entropy Criterion for Moment Indeterminacy of Probability Densities

Jordan M. Stoyanov,Aldo Tagliani,Pier Luigi Novi Inverardi

DOI: https://doi.org/10.3390/e26020121

IF: 2.738

2024-01-31

Entropy

Abstract:We deal with absolutely continuous probability distributions with finite all-positive integer-order moments. It is well known that any such distribution is either uniquely determined by its moments (M-determinate), or it is non-unique (M-indeterminate). In this paper, we follow the maximum entropy approach and establish a new criterion for the M-indeterminacy of distributions on the positive half-line (Stieltjes case). Useful corollaries are derived for M-indeterminate distributions on the whole real line (Hamburger case). We show how the maximum entropy is related to the symmetry property and the M-indeterminacy.

physics, multidisciplinary
Nonparametric maximum likelihood estimation under a likelihood ratio order

Ted Westling,Kevin J. Downes,Dylan S. Small

DOI: https://doi.org/10.48550/arXiv.1904.12321

2021-07-07

Abstract:Comparison of two univariate distributions based on independent samples from them is a fundamental problem in statistics, with applications in a wide variety of scientific disciplines. In many situations, we might hypothesize that the two distributions are stochastically ordered, meaning intuitively that samples from one distribution tend to be larger than those from the other. One type of stochastic order that arises in economics, biomedicine, and elsewhere is the likelihood ratio order, also known as the density ratio order, in which the ratio of the density functions of the two distributions is monotone non-decreasing. In this article, we derive and study the nonparametric maximum likelihood estimator of the individual distributions and the ratio of their densities under the likelihood ratio order. Our work applies to discrete distributions, continuous distributions, and mixed continuous-discrete distributions. We demonstrate convergence in distribution of the estimator in certain cases, and we illustrate our results using numerical experiments and an analysis of a biomarker for predicting bacterial infection in children with systemic inflammatory response syndrome.

Methodology,Statistics Theory
Minimax properties of Dirichlet kernel density estimators

Karine Bertin,Christian Genest,Nicolas Klutchnikoff,Frédéric Ouimet

DOI: https://doi.org/10.1016/j.jmva.2023.105158

2023-01-31

Abstract:This paper considers the asymptotic behavior in $\beta$-Hölder spaces, and under $L^p$ losses, of a Dirichlet kernel density estimator proposed by Aitchison and Lauder (1985) for the analysis of compositional data. In recent work, Ouimet and Tolosana-Delgado (2022) established the uniform strong consistency and asymptotic normality of this estimator. As a complement, it is shown here that the Aitchison-Lauder estimator can achieve the minimax rate asymptotically for a suitable choice of bandwidth whenever $(p,\beta) \in [1, 3) \times (0, 2]$ or $(p, \beta) \in \mathcal{A}_d$, where $\mathcal{A}_d$ is a specific subset of $[3, 4) \times (0, 2]$ that depends on the dimension $d$ of the Dirichlet kernel. It is also shown that this estimator cannot be minimax when either $p \in [4, \infty)$ or $\beta \in (2, \infty)$. These results extend to the multivariate case, and also rectify in a minor way, earlier findings of Bertin and Klutchnikoff (2011) concerning the minimax properties of Beta kernel estimators.

Statistics Theory,Probability
Varieties with maximum likelihood degree one

June Huh

DOI: https://doi.org/10.18409/jas.v5i1.22

2014-04-30

Journal of Algebraic Statistics

Abstract:We show that algebraic varieties with maximum likelihood degree one are exactlythe images of reduced A-discriminantal varieties under monomial maps with nite bers. Themaximum likelihood estimator corresponding to such a variety is Kapranov's Horn uniformization.This extends Kapranov's characterization of A-discriminantal hypersurfaces to varieties of arbitrarycodimension.
Generalized multi-view model: Adaptive density estimation under low-rank constraints

Julien Chhor,Olga Klopp,Alexandre Tsybakov

2024-10-23

Abstract:We study the problem of bivariate discrete or continuous probability density estimation under low-rank <a class="link-external link-http" href="http://constraints.For" rel="external noopener nofollow">this http URL</a> discrete distributions, we assume that the two-dimensional array to estimate is a low-rank probability matrix. In the continuous case, we assume that the density with respect to the Lebesgue measure satisfies a generalized multi-view model, meaning that it is $\beta$-H{ö}lder and can be decomposed as a sum of $K$ components, each of which is a product of one-dimensional functions. In both settings, we propose estimators that achieve, up to logarithmic factors, the minimax optimal convergence rates under such low-rank constraints. In the discrete case, the proposed estimator is adaptive to the rank $K$. In the continuous case, our estimator converges with the $L_1$ rate $\min((K/n)^{\beta/(2\beta+1)}, n^{-\beta/(2\beta+2)})$ up to logarithmic factors, and it is adaptive to the unknown support as well as to the smoothness $\beta$ and to the unknown number of separable components $K$. We present efficient algorithms for computing our estimators.

Statistics Theory
Convergence rates for estimating multivariate scale mixtures of uniform densities

Arlene K. H. Kim,Gil Kur,Adityanand Guntuboyina

2024-10-14

Abstract:The Grenander estimator is a well-studied procedure for univariate nonparametric density estimation. It is usually defined as the Maximum Likelihood Estimator (MLE) over the class of all non-increasing densities on the positive real line. It can also be seen as the MLE over the class of all scale mixtures of uniform densities. Using the latter viewpoint, Pavlides and Wellner~\cite{pavlides2012nonparametric} proposed a multivariate extension of the Grenander estimator as the nonparametric MLE over the class of all multivariate scale mixtures of uniform densities. We prove that this multivariate estimator achieves the univariate cube root rate of convergence with only a logarithmic multiplicative factor that depends on the dimension. The usual curse of dimensionality is therefore avoided to some extent for this multivariate estimator. This result positively resolves a conjecture of Pavlides and Wellner~\cite{pavlides2012nonparametric} under an additional lower bound assumption. Our proof proceeds via a general accuracy result for the Hellinger accuracy of MLEs over convex classes of densities.

Statistics Theory
Uniform Convergence Rate of the Kernel Density Estimator Adaptive to Intrinsic Volume Dimension

Jisu Kim,Jaehyeok Shin,Alessandro Rinaldo,Larry Wasserman

DOI: https://doi.org/10.48550/arXiv.1810.05935

2019-12-31

Abstract:We derive concentration inequalities for the supremum norm of the difference between a kernel density estimator (KDE) and its point-wise expectation that hold uniformly over the selection of the bandwidth and under weaker conditions on the kernel and the data generating distribution than previously used in the literature. We first propose a novel concept, called the volume dimension, to measure the intrinsic dimension of the support of a probability distribution based on the rates of decay of the probability of vanishing Euclidean balls. Our bounds depend on the volume dimension and generalize the existing bounds derived in the literature. In particular, when the data-generating distribution has a bounded Lebesgue density or is supported on a sufficiently well-behaved lower-dimensional manifold, our bound recovers the same convergence rate depending on the intrinsic dimension of the support as ones known in the literature. At the same time, our results apply to more general cases, such as the ones of distribution with unbounded densities or supported on a mixture of manifolds with different dimensions. Analogous bounds are derived for the derivative of the KDE, of any order. Our results are generally applicable but are especially useful for problems in geometric inference and topological data analysis, including level set estimation, density-based clustering, modal clustering and mode hunting, ridge estimation and persistent homology.

Statistics Theory
Uniqueness of Convex-Ranged Probabilities and Applications to Risk Measures and Games

Massimiliano Amarante,Felix-Benedikt Liebrich,Cosimo Munari

DOI: https://doi.org/10.1287/moor.2023.0015

IF: 2.215

2024-03-29

Mathematics of Operations Research

Abstract:We revisit Marinacci’s uniqueness theorem for convex-ranged probabilities and its applications. Our approach does away with both the countable additivity and the positivity of the charges involved. In the process, we uncover several new equivalent conditions, which lead to a novel set of applications. These include extensions of the classic Fréchet–Hoeffding bounds as well as of the automatic Fatou property of law-invariant functionals. We also generalize existing results of the “collapse to the mean”-type concerning capacities and α-MEU preferences.

mathematics, applied,operations research & management science
Maximal point-polyserial correlation for non-normal random distributions

Alessandro Barbiero

DOI: https://doi.org/10.1111/bmsp.12362

2024-10-22

Abstract:We consider the problem of determining the maximum value of the point-polyserial correlation between a random variable with an assigned continuous distribution and an ordinal random variable with k $$ k $$ categories, which are assigned the first k $$ k $$ natural values 1 , 2 , … , k $$ 1,2,\dots, k $$ , and arbitrary probabilities p i $$ {p}_i $$ . For different parametric distributions, we derive a closed-form formula for the maximal point-polyserial correlation as a function of the p i $$ {p}_i $$ and of the distribution's parameters; we devise an algorithm for obtaining its maximum value numerically for any given k $$ k $$ . These maximum values and the features of the corresponding k $$ k $$ -point discrete random variables are discussed with respect to the underlying continuous distribution. Furthermore, we prove that if we do not assign the values of the ordinal random variable a priori but instead include them in the optimization problem, this latter approach is equivalent to the optimal quantization problem. In some circumstances, it leads to a significant increase in the maximum value of the point-polyserial correlation. An application to real data exemplifies the main findings. A comparison between the discretization leading to the maximum point-polyserial correlation and those obtained from optimal quantization and moment matching is sketched.
A mixture of a normal distribution with random mean and variance -- Examples of inconsistency of maximum likelihood estimates

Ya'acov Ritov

2024-08-17

Abstract:We consider the estimation of the mixing distribution of a normal distribution where both the shift and scale are unobserved random variables. We argue that in general, the model is not identifiable. We give an elegant non-constructive proof that the model is identifiable if the shift parameter is bounded by a known value. However, we argue that the generalized maximum likelihood estimator is inconsistent even if the shift parameter is bounded and the shift and scale parameters are independent. The mixing distribution, however, is identifiable if we have more than one observations per any realization of the latent shift and scale.

Statistics Theory
Convolutions of Totally Positive Distributions with applications to Kernel Density Estimation

Ali Zartash,Elina Robeva

DOI: https://doi.org/10.2140/astat.2022.13.57

2020-12-27

Abstract:In this work we study the estimation of the density of a totally positive random vector. Total positivity of the distribution of a random vector implies a strong form of positive dependence between its coordinates and, in particular, it implies positive association. Since estimating a totally positive density is a non-parametric problem, we take on a (modified) kernel density estimation approach. Our main result is that the sum of scaled standard Gaussian bumps centered at a min-max closed set provably yields a totally positive distribution. Hence, our strategy for producing a totally positive estimator is to form the min-max closure of the set of samples, and output a sum of Gaussian bumps centered at the points in this set. We can frame this sum as a convolution between the uniform distribution on a min-max closed set and a scaled standard Gaussian. We further conjecture that convolving any totally positive density with a standard Gaussian remains totally positive.

Mathematics
Maximum likelihood estimators uniformly minimize distribution variance among distribution unbiased estimators in exponential families

Paul Vos,Qiang Wu

DOI: https://doi.org/10.3150/14-BEJ637

2015-09-18

Abstract:We employ a parameter-free distribution estimation framework where estimators are random distributions and utilize the Kullback-Leibler (KL) divergence as a loss function. Wu and Vos [J. Statist. Plann. Inference 142 (2012) 1525-1536] show that when an estimator obtained from an i.i.d. sample is viewed as a random distribution, the KL risk of the estimator decomposes in a fashion parallel to the mean squared error decomposition when the estimator is a real-valued random variable. In this paper, we explore how conditional versions of distribution expectation ($E^{\dagger}$) can be defined so that a distribution version of the Rao-Blackwell theorem holds. We define distributional expectation and variance ($V^{\dagger}$) that also provide a decomposition of KL risk in exponential and mixture families. For exponential families, we show that the maximum likelihood estimator (viewed as a random distribution) is distribution unbiased and is the unique uniformly minimum distribution variance unbiased (UMV$^{\dagger}$U) estimator. Furthermore, we show that the MLE is robust against model specification in that if the true distribution does not belong to the exponential family, the MLE is UMV$^{\dagger}$U for the KL projection of the true distribution onto the exponential families provided these two distribution have the same expectation for the canonical statistic. To allow for estimators taking values outside of the exponential family, we include results for KL projection and define an extended projection to accommodate the non-existence of the MLE for families having discrete sample space. Illustrative examples are provided.

Statistics Theory

Uniqueness of the maximum likelihood estimator for k-monotone densities

Limit theorems for kernel density estimators under dependent samples

On the Maximum Likelihood Estimators' Uniqueness and Existence for Two Unitary Distributions: Analytically and Graphically, with Application

Bayesian Inference for $k$-Monotone Densities with Applications to Multiple Testing

A UNIFIED STUDY OF NONPARAMETRIC INFERENCE FOR MONOTONE FUNCTIONS

On uniqueness of the set of k-means

Bayesian Inference for Multivariate Monotone Densities

A note on the convergence rate of the kernel density estimator of the mode

On the tails of log-concave density estimators

Maximum Entropy Criterion for Moment Indeterminacy of Probability Densities

Nonparametric maximum likelihood estimation under a likelihood ratio order

Minimax properties of Dirichlet kernel density estimators

Varieties with maximum likelihood degree one

Generalized multi-view model: Adaptive density estimation under low-rank constraints

Convergence rates for estimating multivariate scale mixtures of uniform densities

Uniform Convergence Rate of the Kernel Density Estimator Adaptive to Intrinsic Volume Dimension

Uniqueness of Convex-Ranged Probabilities and Applications to Risk Measures and Games

Maximal point-polyserial correlation for non-normal random distributions

A mixture of a normal distribution with random mean and variance -- Examples of inconsistency of maximum likelihood estimates

Convolutions of Totally Positive Distributions with applications to Kernel Density Estimation

Maximum likelihood estimators uniformly minimize distribution variance among distribution unbiased estimators in exponential families