Abstract:Effective vector representation models, e.g., word2vec and node2vec, embed real-world objects such as images and documents in high dimensional vector space. In the meanwhile, the objects are often associated with attributes such as timestamps and prices. Many scenarios need to jointly query the vector representations of the objects together with their attributes. These queries can be formalized as range-filtering approximate nearest neighbor search (ANNS) queries. Specifically, given a collection of data vectors, each associated with an attribute value whose domain has a total order. The range-filtering ANNS consists of a query range and a query vector. It finds the approximate nearest neighbors of the query vector among all the data vectors whose attribute values fall in the query range. Existing approaches suffer from a rapidly degrading query performance when the query range width shifts. The query performance can be optimized by a solution that builds an ANNS index for every possible query range; however, the index time and index size become prohibitive -- the number of query ranges is quadratic to the number n of data vectors. To overcome these challenges, for the query range contains all attribute values smaller than a user-provided threshold, we design a structure called the segment graph whose index time and size are the same as a single ANNS index, yet can losslessly compress the n ANNS indexes, reducing the indexing cost by a factor of Ω(n). To handle general range queries, we propose a 2D segment graph with average-case index size O(n log n) to compress n segment graphs, breaking the quadratic barrier. Extensive experiments conducted on real-world datasets show that our proposed structures outperformed existing methods significantly; our index also exhibits superior scalability.

Succinct Range Filters

SuRF: Practical Range Query Filtering with Fast Succinct Tries.

RoBF: an Auto-Tuning Bloom Filter for Mixed Queries on LSM-Tree.

GRF: A Global Range Filter for LSM-Trees with Shape Encoding

REncoder: A Space-Time Efficient Range Filter with Local Encoder.

Enabling Space-Time Efficient Range Queries with REncoder

Proteus: A Self-Designing Range Filter

iRangeGraph: Improvising Range-dedicated Graphs for Range-filtering Nearest Neighbor Search

Oasis: an Optimal Disjoint Segmented Learned Range Filter

Memento Filter: A Fast, Dynamic, and Robust Range Filter

Bf-Tree: A Modern Read-Write-Optimized Concurrent Larger-Than-Memory Range Index

UNIFY: Unified Index for Range Filtered Approximate Nearest Neighbors Search

ElasticBF: Fine-grained and Elastic Bloom Filter Towards Efficient Read for LSM-tree-based KV Stores.

Tagged Cuckoo Filters

Additive and Subtractive Cuckoo Filters

Don't Thrash: How to Cache Your Hash on Flash

SeRF: Segment Graph for Range-Filtering Approximate Nearest Neighbor Search

SaveRF: Towards Efficient Relevance Feedback Search

Set-Difference Range Queries

A Secure and Fast Range Query Scheme for Encrypted Multi-Dimensional Data