Abstract:In large-scale machine learning, of central interest is the problem of approximate nearest neighbor (ANN) search, where the goal is to query particular points that are close to a given object under certain metric. In this paper, we develop a novel data-driven ANN search algorithm where the data structure is learned by fast spectral technique based on s landmarks selected by approximate ridge leverage scores. We show that with overwhelming probability, our algorithm returns the (1+ϵ/4)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$(1+\epsilon /4)$$\end{document}-ANN for any approximation parameter ϵ∈(0,1)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\epsilon \in (0,1)$$\end{document}. A remarkable feature of our algorithm is that it is computationally efficient. Specifically, learning k-length hash codes requires O((s3+ns2)logn)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$O((s^3+ns^2)\log n)$$\end{document} running time and O(d2)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$O(d^2)$$\end{document} extra space, and returning the (1+ϵ/4)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$(1+\epsilon /4)$$\end{document}-ANN of the query needs O(klogn)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$O(k\log n)$$\end{document} running time. The experimental results on computer vision and natural language understanding tasks demonstrate the significant advantage of our algorithm compared to state-of-the-art methods.

Effective and General Distance Computation for Approximate Nearest Neighbor Search

Accelerating Exact Nearest Neighbor Search in High Dimensional Euclidean Space Via Block Vectors

Efficient Data-aware Distance Comparison Operations for High-Dimensional Approximate Nearest Neighbor Search

Constrained All-k-Nearest-Neighbor Search

Fast High-dimensional Approximate Nearest Neighbor Search with Efficient Index Time and Space

Approximate Nearest Neighbor Search on High Dimensional Data — Experiments, Analyses, and Improvement

Efficient Approximate Algorithms for the Closest Pair Problem in High Dimensional Spaces.

An Efficient Exact Nearest Neighbor Search By Compounded Embedding

Efficient Nearest Neighbor Search in High Dimensional Hamming Space

Doubly Approximate Nearest Neighbor Classification

Preserving-Ignoring Transformation Based Index for Approximate k Nearest Neighbor Search

Random Projection with Filtering for Nearly Duplicate Search

SRS: solving c-approximate nearest neighbor queries in high dimensional euclidean space with a tiny index

Effective optimizations of cluster-based nearest neighbor search in high-dimensional space

Near-Optimal Partial Linear Scan for Nearest Neighbor Search in High-Dimensional Space.

Efficient Metric All-k-Nearest-Neighbor Search on Datasets Without Any Index

Efficient approximation of Earth Mover's Distance Based on Nearest Neighbor Search

Fast Approximate Nearest Neighbor Search Via K-Diverse Nearest Neighbor Graph.

Fast spectral analysis for approximate nearest neighbor search

An Efficient Pre-Computation Technique for Approximation KNN Search in Road Networks.

Approximate nearest neighbor algorithm based on navigable small world graphs