Abstract:Similarity search plays an important role in many applications involving high-dimensional data. Due to the known dimensionality curse, the performance of most existing indexing structures degrades quickly as the feature dimensionality increases. Hashing methods, such as locality sensitive hashing (LSH) and its variants, have been widely used to achieve fast approximate similarity search by trading search quality for efficiency. However, most existing hashing methods make use of randomized algorithms to generate hash codes without considering the specific structural information in the data. In this paper, we propose a novel hashing method, namely, robust hashing with local models (RHLM), which learns a set of robust hash functions to map the high-dimensional data points into binary hash codes by effectively utilizing local structural information. In RHLM, for each individual data point in the training dataset, a local hashing model is learned and used to predict the hash codes of its neighboring data points. The local models from all the data points are globally aligned so that an optimal hash code can be assigned to each data point. After obtaining the hash codes of all the training data points, we design a robust method by employing l2,1 -norm minimization on the loss function to learn effective hash functions, which are then used to map each database point into its hash code. Given a query data point, the search process first maps it into the query hash code by the hash functions and then explores the buckets, which have similar hash codes to the query hash code. Extensive experimental results conducted on real-life datasets show that the proposed RHLM outperforms the state-of-the-art methods in terms of search quality and efficiency.

Exact and Approximate Maximum Inner Product Search with LEMP

Accelerating Exact Nearest Neighbor Search in High Dimensional Euclidean Space Via Block Vectors

ProMIPS: Efficient High-Dimensional C-Approximate Maximum Inner Product Search with a Lightweight Index

Clustering is Efficient for Approximate Maximum Inner Product Search

Lazylsh: Approximate Nearest Neighbor Search For Multiple Distance Functions With A Single Index

FARGO: Fast Maximum Inner Product Search Via Global Multi-Probing.

Efficient Approximate Algorithms for the Closest Pair Problem in High Dimensional Spaces.

Efficient Approximate Search for Sets of Vectors

Preserving-Ignoring Transformation Based Index for Approximate k Nearest Neighbor Search

A Learned Index for Exact Similarity Search in Metric Spaces

Efficient Retrieval with Learned Similarities

Experimental Analysis of Locality Sensitive Hashing Techniques for High-Dimensional Approximate Nearest Neighbor Searches

PM-LSH

Relevance Filtering for Embedding-based Retrieval

Experimental Analysis of Machine Learning Techniques for Finding Search Radius in Locality Sensitive Hashing

DET-LSH: A Locality-Sensitive Hashing Scheme with Dynamic Encoding Tree for Approximate Nearest Neighbor Search

Scalability and Total Recall with Fast CoveringLSH

On the Complexity of Inner Product Similarity Join

Efficient Nearest Neighbor Search in High Dimensional Hamming Space

A Greedy Approach for Budgeted Maximum Inner Product Search

Robust hashing with local models for approximate similarity search.