Abstract:In this paper we revisit the kernel density estimation problem: given a kernel $K(x, y)$ and a dataset of $n$ points in high dimensional Euclidean space, prepare a data structure that can quickly output, given a query $q$, a ($1+ \epsilon$)-approximation to $\mu:=\frac{1}{\vert P\vert}\sum\nolimits_{p\in P}K(p, q)$. First, we give a single data structure based on classical near neighbor search techniques that improves upon or essentially matches the query time and space complexity for all radial kernels considered in the literature so far. We then show how to improve both the query complexity and runtime by using recent advances in data-dependent near neighbor search. We achieve our results by giving an new implementation of the natural importance sampling scheme. Unlike previous approaches, our algorithm first samples the dataset uniformly (considering a geometric sequence of sampling rates), and then uses existing approximate near neighbor search techniques on the resulting smaller dataset to retrieve the sampled points that lie at an appropriate distance from the query. We show that the resulting sampled dataset has strong geometric structure, making approximate near neighbor search return the required samples much more efficiently than for worst case datasets of the same size. As an example application, we show that this approach yields a data structure that achieves query time and space complexity for the Gaussian kernel. Our data dependent approach achieves query time and space for the Gaussian kernel. The data dependent analysis relies on new techniques for tracking the geometric structure of the input datasets in a recursive hashing process that we hope will be of interest in other applications in near neighbor search.

LiNearN: A New Approach to Nearest Neighbour Density Estimator

Accelerating Exact Nearest Neighbor Search in High Dimensional Euclidean Space Via Block Vectors

Demass: A New Density Estimator for Big Data

Density Estimation Based on Mass

A novel density-based clustering algorithm using nearest neighbor graph

An Efficient Exact Nearest Neighbor Search By Compounded Embedding

Lazylsh: Approximate Nearest Neighbor Search For Multiple Distance Functions With A Single Index

Kernel Density Estimation through Density Constrained Near Neighbor Search

A New Simple and Efficient Density Estimator That Enables Fast Systematic Search

Nearest Feature Line Classifier Based on Collaborative Representation with Nearest Neighbour Search Algorithm

Neighbourhood Contrast: A Better Means to Detect Clusters Than Density.

Fast k-Nearest Neighbour Search via Dynamic Continuous Indexing

Efficient Nearest Neighbor Search Using Dynamic Programming

A Novel Density Peaks Clustering Algorithm Based on K Nearest Neighbors with Adaptive Merging Strategy

D-NND: A Hierarchical Density Clustering Method Via Nearest Neighbor Descent

Nearest Neighbor Dirichlet Mixtures

Density Sensitive Hashing

HDIdx: High-dimensional indexing for efficient approximate nearest neighbor search

Approximate Nearest Neighbor Search on High Dimensional Data — Experiments, Analyses, and Improvement

iDEC: indexable distance estimating codes for approximate nearest neighbor search

Lowest Probability Mass Neighbour Algorithms: Relaxing the Metric Constraint in Distance-Based Neighbourhood Algorithms