Abstract:In this paper we revisit the kernel density estimation problem: given a kernel $K(x, y)$ and a dataset of $n$ points in high dimensional Euclidean space, prepare a data structure that can quickly output, given a query $q$, a ($1+ \epsilon$)-approximation to $\mu:=\frac{1}{\vert P\vert}\sum\nolimits_{p\in P}K(p, q)$. First, we give a single data structure based on classical near neighbor search techniques that improves upon or essentially matches the query time and space complexity for all radial kernels considered in the literature so far. We then show how to improve both the query complexity and runtime by using recent advances in data-dependent near neighbor search. We achieve our results by giving an new implementation of the natural importance sampling scheme. Unlike previous approaches, our algorithm first samples the dataset uniformly (considering a geometric sequence of sampling rates), and then uses existing approximate near neighbor search techniques on the resulting smaller dataset to retrieve the sampled points that lie at an appropriate distance from the query. We show that the resulting sampled dataset has strong geometric structure, making approximate near neighbor search return the required samples much more efficiently than for worst case datasets of the same size. As an example application, we show that this approach yields a data structure that achieves query time and space complexity for the Gaussian kernel. Our data dependent approach achieves query time and space for the Gaussian kernel. The data dependent analysis relies on new techniques for tracking the geometric structure of the input datasets in a recursive hashing process that we hope will be of interest in other applications in near neighbor search.

Poisson Kernel-Based Clustering on the Sphere: Convergence Properties, Identifiability, and a Method of Sampling

Kernel density estimation with polyspherical data and its applications

Directional density-based clustering

Parameter Estimation and Model-Based Clustering with Spherical Normal Distribution on the Unit Hypersphere

The geometry of kernelized spectral clustering

Geodesic slice sampling on the sphere

A Statistical Information-Based Clustering Approach in Distance Space

Clustering and Geodesic Scaling of Dissimilarities on the Spherical Surface

Concentration of Measure for Radial Distributions and Consequences for Statistical Modeling

Statistical Inference on the Hilbert Sphere with Application to Random Densities

Generalised Watson Distribution on the Hypersphere with Applications to Clustering.

The cylindrical K-function and Poisson line cluster point processes

Kernel Density Estimation through Density Constrained Near Neighbor Search

Goodness-of-Fit and Clustering of Spherical Data: the QuadratiK package in R and Python

Spectral Clustering on Large Datasets: When Does it Work? Theory from Continuous Clustering and Density Cheeger-Buser

Spherical random projection

Flexible Clustering with a Sparse Mixture of Generalized Hyperbolic Distributions

Finite Mixtures of Multivariate Wrapped Normal Distributions for Model Based Clustering of p-Torus Data

Morphometry on the sphere: Cartesian and irreducible Minkowski tensors explained and implemented

Algorithms for K-Nearest Neighbor Query on Sphere

Self-Weighted Euler $K$-Means Clustering.