Abstract:Similar text search aims to find texts relevant to a given query from a database, which is fundamental in many information retrieval applications, such as question search and exercise search. Since millions of texts always exist behind practical search engine systems, a well-developed text search system usually consists of recall and ranking stages. Specifically, the recall stage serves as the basis in the system, where the main purpose is to find a small set of relevant candidates accurately and efficiently. Towards this goal, deep semantic hashing, which projects original texts into compact hash codes, can support good search performance. However, learning desired textual hash codes is extremely difficult due to the following problems. First, compact hash codes (with short length) can improve retrieval efficiency, but the demand for learning compact hash codes cannot guarantee accuracy due to severe information loss. Second, existing methods always learn the unevenly distributed codes in the space from a local perspective, leading to unsatisfactory code-balance results. Third, a large fraction of textual data contains various types of noise in real-world applications, which causes the deviation of semantics in hash codes. To this end, in this paper, we first propose a general unsupervised encoder-decoder semantic hashing framework, namely MASH (short for Memory-bAsed Semantic Hashing), to learn the balanced and compact hash codes for similar text search. Specifically, with a target of retaining semantic information as much as possible, the encoder introduces a novel relevance constraint among informative high-dimensional representations to guide the compact hash code learning. Then, we design an external memory where the hashing learning can be optimized in the global space to ensure the code balance of the learning results, which can promote search efficiency. Besides, to alleviate the performance degradation problem of the model caused by text noise, we propose an improved SMASH (short for denoiSing Memory-bAsed Semantic Hashing) model by incorporating a noise-aware encoder-decoder framework. This framework considers the noise degree for each text from the semantic deviation aspect, ensuring the robustness of hash codes. Finally, we conduct extensive experiments in three real-world datasets. The experimental results clearly demonstrate the effectiveness and efficiency of MASH and SMASH in generating balanced and compact hash codes, as well as the superior denoising ability of SMASH.

Large Scale Sentiment Analysis with Locality Sensitive BitHash.

BitHash: an Efficient Bitwise Locality Sensitive Hashing Method with Applications

Online latent semantic hashing for cross-media retrieval.

Supervised Multi-scale Locality Sensitive Hashing

Supervised Hashing With Pseudo Labels For Scalable Multimedia Retrieval

Accelerated Manhattan Hashing Via Bit-Remapping with Location Information.

A Scalable, Lexicon Based Technique for Sentiment Analysis

Supervised Short-Length Hashing.

Semi-Supervised Hashing With Semantic Confidence For Large Scale Visual Search

A Mixed Generative-Discriminative Based Hashing Method

An Efficient and Robust Semantic Hashing Framework for Similar Text Search

Seg-Hashnet: Semantic Segmentation Based Unsupervised Hashing

Compressed Hashing

Collaborative Learning for Extremely Low Bit Asymmetric Hashing

Learning-based distributed locality sensitive hashing.

Piecewise Hashing: A Deep Hashing Method for Large-Scale Fine-Grained Search

Topic Sentiment Analysis in Twitter

Scalable deep asymmetric hashing via unequal-dimensional embeddings for image similarity search

Self-supervised asymmetric deep hashing with margin-scalable constraint

Supervised Hashing With Latent Factor Models

Super-Bit Locality-Sensitive Hashing.