Abstract:In large-scale machine learning, of central interest is the problem of approximate nearest neighbor (ANN) search, where the goal is to query particular points that are close to a given object under certain metric. In this paper, we develop a novel data-driven ANN search algorithm where the data structure is learned by fast spectral technique based on s landmarks selected by approximate ridge leverage scores. We show that with overwhelming probability, our algorithm returns the (1+ϵ/4)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$(1+\epsilon /4)$$\end{document}-ANN for any approximation parameter ϵ∈(0,1)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\epsilon \in (0,1)$$\end{document}. A remarkable feature of our algorithm is that it is computationally efficient. Specifically, learning k-length hash codes requires O((s3+ns2)logn)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$O((s^3+ns^2)\log n)$$\end{document} running time and O(d2)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$O(d^2)$$\end{document} extra space, and returning the (1+ϵ/4)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$(1+\epsilon /4)$$\end{document}-ANN of the query needs O(klogn)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$O(k\log n)$$\end{document} running time. The experimental results on computer vision and natural language understanding tasks demonstrate the significant advantage of our algorithm compared to state-of-the-art methods.

Accelerating Exact Nearest Neighbor Search in High Dimensional Euclidean Space Via Block Vectors

Efficient Approximate Algorithms for the Closest Pair Problem in High Dimensional Spaces.

Approximate Nearest Neighbor Search on High Dimensional Data — Experiments, Analyses, and Improvement

Fast High-dimensional Approximate Nearest Neighbor Search with Efficient Index Time and Space

Effective and General Distance Computation for Approximate Nearest Neighbor Search

Lazylsh: Approximate Nearest Neighbor Search For Multiple Distance Functions With A Single Index

Efficient Nearest Neighbor Search in High Dimensional Hamming Space

A scalable solution to the nearest neighbor search problem through local-search methods on neighbor graphs

Efficient Data-aware Distance Comparison Operations for High-Dimensional Approximate Nearest Neighbor Search

Worst-case Performance of Popular Approximate Nearest Neighbor Search Implementations: Guarantees and Limitations

Large Scale Nearest Neighbors Search Based on Neighborhood Graph.

Group Testing for Accurate and Efficient Range-Based Near Neighbor Search : An Adaptive Binary Splitting Approach

Fast Nearest Neighbor Search in the Hamming Space.

HDIdx: High-dimensional indexing for efficient approximate nearest neighbor search

Random Angular Projection for Fast Nearest Subspace Search

Preserving-Ignoring Transformation Based Index for Approximate k Nearest Neighbor Search

Composite Distance Transformation for Indexing and K -Nearest-neighbor Searching in High-Dimensional Spaces

Experimental Analysis of Locality Sensitive Hashing Techniques for High-Dimensional Approximate Nearest Neighbor Searches

Random Projection with Filtering for Nearly Duplicate Search

Speeding Up K-Means Clustering in High Dimensions by Pruning Unnecessary Distance Computations

Fast spectral analysis for approximate nearest neighbor search