Abstract:Similarity search is crucial to many applications. Of particular interest are two flavors of the Hamming distance range query, namely, the Hamming select and the Hamming join (Hamming-select and Hamming-join, respectively). Hamming distance is widely used in approximate near neighbor search for high dimensional data, such as images and document collections. For example, using predefined similarity hash functions, high-dimensional data is mapped into one-dimensional binary codes that are, then linearly scanned to perform Hamming-distance comparisons. These distance comparisons on the binary codes are usually costly and, often involves excessive redundancies. This paper introduces a new index, termed the HA-Index, that speeds up distance comparisons and eliminates redundancies when performing the two flavors of Hamming distance range queries. An efficient search algorithm based on the HA-index is presented. A distributed version of the HA-index is introduced and algorithms for realizing Hamming distance-select and Hamming distance-join operations on a MapReduce platform are prototyped. Extensive experiments using real datasets demonstrates that the HA-index and the corresponding search algorithms achieve up to two orders of magnitude speedup over existing stateof-the-art approaches, while saving more than ten times in memory space.

A Text Similarity Simulation Detection Model Based on Hadoop

Batch Text Similarity Search with MapReduce

Parallel Simrank Computing on Large Scale Dataset on Mapreduce

Similarity Measure Based on Improved Optimal Assignment Model

HadoopM: A Message-Enabled Data Processing System on Large Clusters.

A Fast Document Copy Detection Model

Efficient Spatio-textual Similarity Join Using MapReduce

Using Link-Based Content Analysis to Measure Document Similarity Effectively

A Novel Cluster Computing Technique Based on Signal Clustering and Analytic Hierarchy Model Using Hadoop

Application of the maintenance text data of transformers based on SimHash and Hamming distance algorithm

A MapReduce based distributed LSI

Text Similarity Measurement of Semantic Cognition Based on Word Vector Distance Decentralization with Clustering Analysis

A New Similarity Computing Method Based on Concept Similarity in Chinese Text Processing.

Efficient Processing of Hamming-Distance-Based Similarity-Search Queries over MapReduce.

Hsim: A Novel Method on Similarity Computation by Hybrid Measure

Parallel Text Categorization of Massive Text Based on Hadoop

Identification of Seed Users Via Short Messages Based on Hadoop

Similarity algorithm of text based on semantic understanding

Using Multiple Features and Statistical Model to Calculate Text Units Similarity

Research on applicability of sentence similarity algorithms in text copy detection

New method of hybrid intelligent text clustering based on semantic similarity