Abstract:We study the string similarity search problem with edit-distance constraints, which, given a set of data strings and a query string, finds the similar strings to the query. Existing algorithms use a signature-based framework. They first generate signatures for each string and then prune the dissimilar strings which have no common signatures to the query. However existing methods involve large numbers of signatures and many signatures are unnecessary. Reducing the number of signatures not only increases the pruning power but also decreases the filtering cost. To address this problem, we propose a novel pivotal prefix filter which significantly reduces the number of signatures. We prove the pivotal filter achieves larger pruning power and less filtering cost than state-of-the-art filters. We develop a dynamic programming method to select high-quality pivotal prefix signatures to prune dissimilar strings with non-consecutive errors to the query. We propose an alignment filter that considers the alignments between signatures to prune large numbers of dissimilar pairs with consecutive errors to the query. Experimental results on three real datasets show that our method achieves high performance and outperforms the state-of-the-art methods by an order of magnitude.

An Approximate String Matching Algorithm for Chinese Information Retrieval Systems

Pinyin-indexed Method for Approximate Matching in Chinese

Algorithm of Multiple Approximate String for Chinese Characters Based on Filtering

A Chinese fuzzy autocompletion approach.

Automatic Chinese text error correction approach based-on fast approximate Chinese word-matching algorithm

Assessment of approximate string matching in a biomedical text retrieval problem

Hierarchical Approximate Matching for Retrieval of Chinese Historical Calligraphy Character

Effective Indices for Efficient Approximate String Search and Similarity Join

A Fast and Accurate Method for Approximate String Search

Improved Word Similarity Computation for Chinese Using Sub-word Information

Fast Approximate Matching Algorithm for Phone-based Keyword Spotting

Towards interactive indexing for large Chinese calligraphic character databases.

On Implementation of Improved Karp-Rabin Pattern Matching Algorighmic

Combining multiple sources for short query translation in Chinese-English cross-language information retrieval

Chinese Error Correction of Searching Engine under N-Gram Statistic Model

English-Chinese Cross-language Information Retrieval Using Lucene System

A Pivotal Prefix Based Filtering Algorithm for String Similarity Search

Study of Word-Based Chinese Document Experimental System and Chinese Free-Text Information Extraction Experiment Based on It

Robust Quick String Matching Algorithm for Network Security

Research on Chinese-English Cross-Language Information Retrieval

Recursive Shift Indexing : A Fast Multi-Pattern String Matching Algorithm