Abstract:The approximate nearest neighbor (ANN) search in high-dimensional spaces is a fundamental but computationally very expensive problem. Many methods have been designed for solving the ANN problem, such as LSH-based methods and graph-based methods. The LSH-based methods can be costly to reach high query quality due to the hash-boundary issues, while the graph-based methods can achieve better query performance by greedy expansion in an approximate proximity graph (APG). However, the construction cost of these APGs can be one or two orders of magnitude higher than that for building hash-based indexes. In addition, they fail short in incrementally maintaining APGs as the underlying dataset evolves. In this paper, we propose a novel approach named LSH-APG to build APGs and facilitate fast ANN search using a lightweight LSH framework. LSH-APG builds an APG via consecutively inserting points based on their nearest neighbor relationship with an efficient and accurate LSH-based search strategy. A high-quality entry point selection technique and an LSH-based pruning condition are developed to accelerate index construction and query processing by reducing the number of points to be accessed during the search. LSH-APG supports fast maintenance of APGs in lieu of building them from scratch as dataset evolves. Its maintenance cost and query cost for a point is proven to be less affected by dataset cardinality. Extensive experiments on real-world and synthetic datasets demonstrate that LSH-APG incurs significantly less construction cost but achieves better query performance than existing graph-based methods.

HDIdx: High-dimensional indexing for efficient approximate nearest neighbor search

Accelerating Exact Nearest Neighbor Search in High Dimensional Euclidean Space Via Block Vectors

Approximate Nearest Neighbor Search on High Dimensional Data — Experiments, Analyses, and Improvement

Effective optimizations of cluster-based nearest neighbor search in high-dimensional space

Near-Optimal Partial Linear Scan for Nearest Neighbor Search in High-Dimensional Space.

A Novel High-Dimensional Index Method Based on the Mathematical Features

Efficient Indexing of Binary Lsh for High Dimensional Nearest Neighbor

Ipoc: A Polar Coordinate Based Indexing Method For Nearest Neighbor Search In High Dimensional Space

Fast High-dimensional Approximate Nearest Neighbor Search with Efficient Index Time and Space

Harmonious Hashing

Efficient Nearest Neighbor Search in High Dimensional Hamming Space

Efficient Data-aware Distance Comparison Operations for High-Dimensional Approximate Nearest Neighbor Search

Approximate Nearest Neighbor Search Based On Hierarchical Multi-Index Hashing

Large Scale Nearest Neighbors Search Based on Neighborhood Graph.

Preserving-Ignoring Transformation Based Index for Approximate k Nearest Neighbor Search

FLEX: A Fast and Light-weight Learned Index for kNN Search in High-Dimensional Space

Towards Efficient Index Construction and Approximate Nearest Neighbor Search in High-Dimensional Spaces.

An Efficient Exact Nearest Neighbor Search By Compounded Embedding

SRS: solving c-approximate nearest neighbor queries in high dimensional euclidean space with a tiny index

I/O Efficient Approximate Nearest Neighbour Search Based on Learned Functions.

Efficient index-based KNN join processing for high-dimensional data