Abstract:In large-scale machine learning, of central interest is the problem of approximate nearest neighbor (ANN) search, where the goal is to query particular points that are close to a given object under certain metric. In this paper, we develop a novel data-driven ANN search algorithm where the data structure is learned by fast spectral technique based on s landmarks selected by approximate ridge leverage scores. We show that with overwhelming probability, our algorithm returns the (1+ϵ/4)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$(1+\epsilon /4)$$\end{document}-ANN for any approximation parameter ϵ∈(0,1)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\epsilon \in (0,1)$$\end{document}. A remarkable feature of our algorithm is that it is computationally efficient. Specifically, learning k-length hash codes requires O((s3+ns2)logn)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$O((s^3+ns^2)\log n)$$\end{document} running time and O(d2)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$O(d^2)$$\end{document} extra space, and returning the (1+ϵ/4)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$(1+\epsilon /4)$$\end{document}-ANN of the query needs O(klogn)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$O(k\log n)$$\end{document} running time. The experimental results on computer vision and natural language understanding tasks demonstrate the significant advantage of our algorithm compared to state-of-the-art methods.

Fast Nearest Neighbor Search in the Hamming Space.

Accelerating Exact Nearest Neighbor Search in High Dimensional Euclidean Space Via Block Vectors

Efficient Nearest Neighbor Search in High Dimensional Hamming Space

Fast and Exact Nearest Neighbor Search in Hamming Space on Full-Text Search Engines

Constrained All-k-Nearest-Neighbor Search

Accelerating Search on Binary Codes in Weighted Hamming Space

Efficient approximate nearest neighbor search with integrated binary codes.

Fast kNN Search in Weighted Hamming Space With Multiple Tables

Fast High-dimensional Approximate Nearest Neighbor Search with Efficient Index Time and Space

Fast Search on Binary Codes by Weighted Hamming Distance

Fast, K Nearest Neighbors Search Algorithm Based on Wavelet Transform

Equal-Average Equal-Variance Nearest Neighbor Search Algorithm Based on Hadamard Transform

Fast Nearest-Neighbor Codeword Search Algorithms for Vector Quantization

Polysemous codes

Fast Norm-Ordered Nearest-Neighbor Codeword Search Algorithms For Image Vector Quantization

Neighborhood Voting: A Novel Search Scheme for Hashing.

Double-Bit Quantization and Index Hashing for Nearest Neighbor Search

Distributed Adaptive Binary Quantization for Fast Nearest Neighbor Search.

Fast codevector search scheme for 3D mesh model vector quantisation

Fast Cosine Similarity Search in Binary Space with Angular Multi-Index Hashing

Fast spectral analysis for approximate nearest neighbor search