Abstract:Similar text search aims to find texts relevant to a given query from a database, which is fundamental in many information retrieval applications, such as question search and exercise search. Since millions of texts always exist behind practical search engine systems, a well-developed text search system usually consists of recall and ranking stages. Specifically, the recall stage serves as the basis in the system, where the main purpose is to find a small set of relevant candidates accurately and efficiently. Towards this goal, deep semantic hashing, which projects original texts into compact hash codes, can support good search performance. However, learning desired textual hash codes is extremely difficult due to the following problems. First, compact hash codes (with short length) can improve retrieval efficiency, but the demand for learning compact hash codes cannot guarantee accuracy due to severe information loss. Second, existing methods always learn the unevenly distributed codes in the space from a local perspective, leading to unsatisfactory code-balance results. Third, a large fraction of textual data contains various types of noise in real-world applications, which causes the deviation of semantics in hash codes. To this end, in this paper, we first propose a general unsupervised encoder-decoder semantic hashing framework, namely MASH (short for Memory-bAsed Semantic Hashing), to learn the balanced and compact hash codes for similar text search. Specifically, with a target of retaining semantic information as much as possible, the encoder introduces a novel relevance constraint among informative high-dimensional representations to guide the compact hash code learning. Then, we design an external memory where the hashing learning can be optimized in the global space to ensure the code balance of the learning results, which can promote search efficiency. Besides, to alleviate the performance degradation problem of the model caused by text noise, we propose an improved SMASH (short for denoiSing Memory-bAsed Semantic Hashing) model by incorporating a noise-aware encoder-decoder framework. This framework considers the noise degree for each text from the semantic deviation aspect, ensuring the robustness of hash codes. Finally, we conduct extensive experiments in three real-world datasets. The experimental results clearly demonstrate the effectiveness and efficiency of MASH and SMASH in generating balanced and compact hash codes, as well as the superior denoising ability of SMASH.

Context-Preserving Hashing for Fast Text Classification.

Nonlinear Discrete Cross-Modal Hashing for Visual-Textual Data

Supervised Coarse-to-Fine Semantic Hashing for Cross-Media Retrieval.

Semantic Consistency Hashing for Cross-Modal Retrieval

Discrete Similarity Preserving Hashing for Cross-modal Retrieval.

An Efficient and Robust Semantic Hashing Framework for Similar Text Search

Cross-modal image–text search via Efficient Discrete Class Alignment Hashing

A Mixed Generative-Discriminative Based Hashing Method

Fast discrete cross-modal hashing with semantic consistency

Fast Scalable Supervised Hashing

Discrete Joint Semantic Alignment Hashing for Cross-Modal Image-Text Search

Reinforcing Short-Length Hashing

Fast Discrete Cross-modal Hashing with Regressing from Semantic Labels.

Supervised adaptive similarity consistent latent representation hashing

Fast Unmediated Hashing for Cross-Modal Retrieval.

Sequential Discrete Hashing for Scalable Cross-Modality Similarity Retrieval

Efficient Fine-Grained Visual-Text Search Using Adversarially-Learned Hash Codes

A Hash-Based Hierarchical Algorithm For Massive Text Clustering

Non-transitive Hashing with Latent Similarity Components

Unsupervised Cross-modal Hashing via Semantic Text Mining