Abstract:This paper studies Kernel Density Estimation for a high-dimensional distribution $\rho(x)$. Traditional approaches have focused on the limit of large number of data points $n$ and fixed dimension $d$. We analyze instead the regime where both the number $n$ of data points $y_i$ and their dimensionality $d$ grow with a fixed ratio $\alpha=(\log n)/d$. Our study reveals three distinct statistical regimes for the kernel-based estimate of the density $\hat \rho_h^{\mathcal {D}}(x)=\frac{1}{n h^d}\sum_{i=1}^n K\left(\frac{x-y_i}{h}\right)$, depending on the bandwidth $h$: a classical regime for large bandwidth where the Central Limit Theorem (CLT) holds, which is akin to the one found in traditional approaches. Below a certain value of the bandwidth, $h_{CLT}(\alpha)$, we find that the CLT breaks down. The statistics of $\hat\rho_h^{\mathcal {D}}(x)$ for a fixed $x$ drawn from $\rho(x)$ is given by a heavy-tailed distribution (an alpha-stable distribution). In particular below a value $h_G(\alpha)$, we find that $\hat\rho_h^{\mathcal {D}}(x)$ is governed by extreme value statistics: only a few points in the database matter and give the dominant contribution to the density estimator. We provide a detailed analysis for high-dimensional multivariate Gaussian data. We show that the optimal bandwidth threshold based on Kullback-Leibler divergence lies in the new statistical regime identified in this paper. As known by practitioners, when decreasing the bandwidth a Kernel-estimated estimated changes from a smooth curve to a collections of peaks centred on the data points. Our findings reveal that this general phenomenon is related to sharp transitions between phases characterized by different statistical properties, and offer new insights for Kernel density estimation in high-dimensional settings.

Pointwise Sharp Moderate Deviations for a Kernel Density Estimator

Limit theorems for kernel density estimators under dependent samples

ASYMPTOTIC NORMALITY OF KERNEL ESTIMATES OF A DENSITY FUNCTION UNDER ASSOCIATION DEPENDENCE

Asymptotic Distributions of Innovation Density Estimators in Linear Processes

A Nonparametric Test for the Change of the Density Function under Association

A note on the convergence rate of the kernel density estimator of the mode

Pointwise adaptive kernel density estimation under local approximate differential privacy

Robust Estimation for Kernel Exponential Families with Smoothed Total Variation Distances

Uniform Convergence Rate of the Kernel Density Estimator Adaptive to Intrinsic Volume Dimension

THE CONSISTENCIES FOR THE KERNEL-TYPE DENSITY[2mm］ ESTIMATION INTHE CASE OF NA SAMPLES

Kernel estimation of multivariate cumulative distribution function

Pointwise equidistribution for almost smooth functions with an error rate and Weighted Lévy-Khintchin theorem

Beta Kernel Estimators for Density Functions

Universal kernel-type estimation of random fields

Kernel regression, minimax rates and effective dimensionality: beyond the regular case

On deviation probabilities in non-parametric regression with heavy-tailed noise

Robust Sparse Kernel Density Estimation by Inducing Randomness

Kernel Estimation of Density Level Sets

Uniform Inference for Kernel Density Estimators with Dyadic Data

Kernel Density Estimators in Large Dimensions

Kernel Two-Sample Tests for Manifold Data