Abstract:Among many solutions to the high-dimensional approximate nearest neighbor (ANN) search problem, locality sensitive hashing (LSH) is known for its sub-linear query time and robust theoretical guarantee on query accuracy. Traditional LSH methods can generate a small number of candidates quickly from hash tables but suffer from large index sizes and hash boundary problems. Recent studies to address these issues often incur extra overhead to identify eligible candidates or remove false positives, making query time no longer sub-linear. To address this dilemma, in this paper we propose a novel LSH scheme called DB-LSH which supports efficient ANN search for large high-dimensional datasets. It organizes the projected spaces with multi-dimensional indexes rather than using fixed-width hash buckets. Our approach can significantly reduce the space cost as by avoiding the need to maintain many hash tables for different bucket sizes. During the query phase of DB-LSH, a small number of high-quality candidates can be generated efficiently by dynamically constructing query-based hypercubic buckets with the required widths through index-based window queries. For a dataset of $n$ $d$-dimensional points with approximation ratio $c$, our rigorous theoretical analysis shows that DB-LSH achieves a smaller query cost ${O(n^{\rho^*} d\log n)}$, where ${\rho^*}$ is bounded by ${1/c^{\alpha}}$ while the bound is ${1/c}$ in the existing work. An extensive range of experiments on real-world data demonstrates the superiority of DB-LSH over state-of-the-art methods on both efficiency and accuracy.

A New Spherical Hashing Method in a Low-Dimensional Isotropic Space.

AN image retrieval method based on multiple hyperspheres OC-SVM hashing

Discrete Similarity Preserving Hashing for Cross-modal Retrieval.

Asymmetric Distance for Spherical Hashing

A New Linkless Hierarchical Structure Based on Perfect Hashing

A Fast Online Spherical Hashing Method Based on Data Sampling for Large Scale Image Retrieval

Isotropic Hashing

Harmonious Hashing

Enhanced Locality Sensitive Clustering in High Dimensional Space

Density Sensitive Hashing

Preserving-Ignoring Transformation Based Index for Approximate k Nearest Neighbor Search

Efficient Locality-Sensitive Hashing over High-Dimensional Streaming Data.

DB-LSH: Locality-Sensitive Hashing with Query-based Dynamic Bucketing

Global Hashing System for Fast Image Search.

An Improved Method of Locality Sensitive Hashing for Indexing Large-Scale and High-Dimensional Features

FSpH: Fitted Spectral Hashing for Efficient Similarity Search

An Improved Hash Function Inspired by the Fly Hashing for Near Duplicate Detections

Multilinear Hyperplane Hashing

Distribution-Aware Locality Sensitive Hashing

Geometry and Topology Preserving Hashing for SIFT Feature

Super-Bit Locality-Sensitive Hashing.