Abstract:In this paper we revisit the kernel density estimation problem: given a kernel $K(x, y)$ and a dataset of $n$ points in high dimensional Euclidean space, prepare a data structure that can quickly output, given a query $q$, a ($1+ \epsilon$)-approximation to $\mu:=\frac{1}{\vert P\vert}\sum\nolimits_{p\in P}K(p, q)$. First, we give a single data structure based on classical near neighbor search techniques that improves upon or essentially matches the query time and space complexity for all radial kernels considered in the literature so far. We then show how to improve both the query complexity and runtime by using recent advances in data-dependent near neighbor search. We achieve our results by giving an new implementation of the natural importance sampling scheme. Unlike previous approaches, our algorithm first samples the dataset uniformly (considering a geometric sequence of sampling rates), and then uses existing approximate near neighbor search techniques on the resulting smaller dataset to retrieve the sampled points that lie at an appropriate distance from the query. We show that the resulting sampled dataset has strong geometric structure, making approximate near neighbor search return the required samples much more efficiently than for worst case datasets of the same size. As an example application, we show that this approach yields a data structure that achieves query time and space complexity for the Gaussian kernel. Our data dependent approach achieves query time and space for the Gaussian kernel. The data dependent analysis relies on new techniques for tracking the geometric structure of the input datasets in a recursive hashing process that we hope will be of interest in other applications in near neighbor search.

Near-Linear Time Local Polynomial Nonparametric Estimation with Box Kernels

Adaptive Estimation Of Time-Varying Parameters In Linear Systems

Two-dimensional recursive least squares identification based on local polynomial modeling for batch processes.

Adaptive Bayesian nonparametric regression using kernel mixture of polynomials with application to partial linear model

A new adaptive local polynomial density estimation procedure on complicated domains

Nonparametric Testing under Randomized Sketching

Nearly Time-Optimal Kernelization Algorithms for the Line-Cover Problem with Big Data

Local Polynomial Estimation of Nonparametric Simultaneous Equations Models

Data-dependent kernels in nearly-linear time

Local Linear Forests

Kernel Density Estimation through Density Constrained Near Neighbor Search

Multi-Layer Kernel Machines: Fast and Optimal Nonparametric Regression with Uncertainty Quantification

Nonparametric Estimation via Partial Derivatives

Quick and Simple Kernel Differential Equation Regression Estimators for Data with Sparse Design

Sublinear Time Spectral Density Estimation

Locally Weighted Kernel Partial Least Squares Regression Based on Sparse Nonlinear Features for Virtual Sensing of Nonlinear Time-Varying Processes

High-performance Kernel Machines with Implicit Distributed Optimization and Randomization

Efficient Nearest Neighbor Search Using Dynamic Programming

Fast Private Kernel Density Estimation via Locality Sensitive Quantization

Local Prediction-Powered Inference