Abstract:The approximate nearest neighbor (ANN) search in high-dimensional spaces is a fundamental but computationally very expensive problem. Many methods have been designed for solving the ANN problem, such as LSH-based methods and graph-based methods. The LSH-based methods can be costly to reach high query quality due to the hash-boundary issues, while the graph-based methods can achieve better query performance by greedy expansion in an approximate proximity graph (APG). However, the construction cost of these APGs can be one or two orders of magnitude higher than that for building hash-based indexes. In addition, they fail short in incrementally maintaining APGs as the underlying dataset evolves. In this paper, we propose a novel approach named LSH-APG to build APGs and facilitate fast ANN search using a lightweight LSH framework. LSH-APG builds an APG via consecutively inserting points based on their nearest neighbor relationship with an efficient and accurate LSH-based search strategy. A high-quality entry point selection technique and an LSH-based pruning condition are developed to accelerate index construction and query processing by reducing the number of points to be accessed during the search. LSH-APG supports fast maintenance of APGs in lieu of building them from scratch as dataset evolves. Its maintenance cost and query cost for a point is proven to be less affected by dataset cardinality. Extensive experiments on real-world and synthetic datasets demonstrate that LSH-APG incurs significantly less construction cost but achieves better query performance than existing graph-based methods.

Efficient Nearest Neighbor Search in High Dimensional Hamming Space

Accelerating Exact Nearest Neighbor Search in High Dimensional Euclidean Space Via Block Vectors

Efficient Indexing of Binary Lsh for High Dimensional Nearest Neighbor

Towards Efficient Index Construction and Approximate Nearest Neighbor Search in High-Dimensional Spaces.

Fast Nearest Neighbor Search in the Hamming Space.

Approximate Nearest Neighbor Search on High Dimensional Data — Experiments, Analyses, and Improvement

SRS: solving c-approximate nearest neighbor queries in high dimensional euclidean space with a tiny index

Efficient Approximate Algorithms for the Closest Pair Problem in High Dimensional Spaces.

Effective optimizations of cluster-based nearest neighbor search in high-dimensional space

Efficient Reverse $k$ Approximate Nearest Neighbor Search over High-Dimensional Vectors

Efficient Nearest Neighbor Query Based on Extended B-tree in High-Dimensional Space.

Preserving-Ignoring Transformation Based Index for Approximate k Nearest Neighbor Search

PM-LSH

Sk-Lsh : An Efficient Index Structure For Approximate Nearest Neighbor Search

Experimental Analysis of Locality Sensitive Hashing Techniques for High-Dimensional Approximate Nearest Neighbor Searches

Efficient Approximate Nearest Neighbor Search in Multi-dimensional Databases.

Effective and General Distance Computation for Approximate Nearest Neighbor Search

Sparse Matrix Based Hashing for Approximate Nearest Neighbor Search.

Fast kNN Search in Weighted Hamming Space With Multiple Tables

Near-Optimal Partial Linear Scan for Nearest Neighbor Search in High-Dimensional Space.

Efficient Data-aware Distance Comparison Operations for High-Dimensional Approximate Nearest Neighbor Search