Abstract:Effective vector representation models, e.g., word2vec and node2vec, embed real-world objects such as images and documents in high dimensional vector space. In the meanwhile, the objects are often associated with attributes such as timestamps and prices. Many scenarios need to jointly query the vector representations of the objects together with their attributes. These queries can be formalized as range-filtering approximate nearest neighbor search (ANNS) queries. Specifically, given a collection of data vectors, each associated with an attribute value whose domain has a total order. The range-filtering ANNS consists of a query range and a query vector. It finds the approximate nearest neighbors of the query vector among all the data vectors whose attribute values fall in the query range. Existing approaches suffer from a rapidly degrading query performance when the query range width shifts. The query performance can be optimized by a solution that builds an ANNS index for every possible query range; however, the index time and index size become prohibitive -- the number of query ranges is quadratic to the number n of data vectors. To overcome these challenges, for the query range contains all attribute values smaller than a user-provided threshold, we design a structure called the segment graph whose index time and size are the same as a single ANNS index, yet can losslessly compress the n ANNS indexes, reducing the indexing cost by a factor of Ω(n). To handle general range queries, we propose a 2D segment graph with average-case index size O(n log n) to compress n segment graphs, breaking the quadratic barrier. Extensive experiments conducted on real-world datasets show that our proposed structures outperformed existing methods significantly; our index also exhibits superior scalability.

Distance Comparison Operators for Approximate Nearest Neighbor Search: Exploration and Benchmark

Accelerating Exact Nearest Neighbor Search in High Dimensional Euclidean Space Via Block Vectors

Approximate Nearest Neighbor Search on High Dimensional Data — Experiments, Analyses, and Improvement

A comprehensive survey and experimental comparison of graph-based approximate nearest neighbor search

Lazylsh: Approximate Nearest Neighbor Search For Multiple Distance Functions With A Single Index

Efficient Data-aware Distance Comparison Operations for High-Dimensional Approximate Nearest Neighbor Search

Deep Learning for Approximate Nearest Neighbour Search: A Survey and Future Directions

Preserving-Ignoring Transformation Based Index for Approximate k Nearest Neighbor Search

Towards Efficient Index Construction and Approximate Nearest Neighbor Search in High-Dimensional Spaces.

A Revisit of Hashing Algorithms for Approximate Nearest Neighbor Search

Optimizing Graph-based Approximate Nearest Neighbor Search: Stronger and Smarter

ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithms

Effective and General Distance Computation for Approximate Nearest Neighbor Search

JUNO: Optimizing High-Dimensional Approximate Nearest Neighbour Search with Sparsity-Aware Algorithm and Ray-Tracing Core Mapping

Subspace Collision: An Efficient and Accurate Framework for High-dimensional Approximate Nearest Neighbor Search

Learning-based Query Optimization for Multi-Probe Approximate Nearest Neighbor Search.

I/O Efficient Approximate Nearest Neighbour Search Based on Learned Functions.

Fast Approximate Nearest Neighbor Search with the Navigating Spreading-out Graph.

SeRF: Segment Graph for Range-Filtering Approximate Nearest Neighbor Search

iDistance: An adaptive B+-tree based indexing method for nearest neighbor search

Efficient and Effective Retrieval of Dense-Sparse Hybrid Vectors using Graph-based Approximate Nearest Neighbor Search