Abstract:This paper studies Kernel Density Estimation for a high-dimensional distribution $\rho(x)$. Traditional approaches have focused on the limit of large number of data points $n$ and fixed dimension $d$. We analyze instead the regime where both the number $n$ of data points $y_i$ and their dimensionality $d$ grow with a fixed ratio $\alpha=(\log n)/d$. Our study reveals three distinct statistical regimes for the kernel-based estimate of the density $\hat \rho_h^{\mathcal {D}}(x)=\frac{1}{n h^d}\sum_{i=1}^n K\left(\frac{x-y_i}{h}\right)$, depending on the bandwidth $h$: a classical regime for large bandwidth where the Central Limit Theorem (CLT) holds, which is akin to the one found in traditional approaches. Below a certain value of the bandwidth, $h_{CLT}(\alpha)$, we find that the CLT breaks down. The statistics of $\hat\rho_h^{\mathcal {D}}(x)$ for a fixed $x$ drawn from $\rho(x)$ is given by a heavy-tailed distribution (an alpha-stable distribution). In particular below a value $h_G(\alpha)$, we find that $\hat\rho_h^{\mathcal {D}}(x)$ is governed by extreme value statistics: only a few points in the database matter and give the dominant contribution to the density estimator. We provide a detailed analysis for high-dimensional multivariate Gaussian data. We show that the optimal bandwidth threshold based on Kullback-Leibler divergence lies in the new statistical regime identified in this paper. As known by practitioners, when decreasing the bandwidth a Kernel-estimated estimated changes from a smooth curve to a collections of peaks centred on the data points. Our findings reveal that this general phenomenon is related to sharp transitions between phases characterized by different statistical properties, and offer new insights for Kernel density estimation in high-dimensional settings.

Topological density estimation

Topological Detection of Phenomenological Bifurcations with Unreliable Kernel Densities

Higher-order topological kernels via quantum computation

Highly Efficient Density-Based Topology Optimization Using DCT-based Digital Image Compression

Asymptotics of Lower Dimensional Zero-Density Regions

Applications of the Topological Data Analysis in Real Life

A Generalized DCT Compression Based Density Method for Topology Optimization of 2D and 3D Continua

Applying topological data analysis to local search problems

Differentially Private Topological Data Analysis

Cross-Matched Interval Prevalence of High Dimensional Point Clouds

Quantum topological data analysis via the estimation of the density of states

Density Estimation Trees in High Energy Physics

Tensor-Train Density Estimation

Bump hunting by topological data analysis

The topology of data hides in quantum thermal states

Uniform Inference for Kernel Density Estimators with Dyadic Data

Chromatic Topological Data Analysis

Kernel Density Estimators in Large Dimensions

Fast Private Kernel Density Estimation via Locality Sensitive Quantization

Probabilistic PCA for T Distributions.

Uniform Convergence Rate of the Kernel Density Estimator Adaptive to Intrinsic Volume Dimension