Abstract:As the demand for global information increases significantly, multilingual corpora has become a valuable linguistic resource for applications to cross‐lingual information retrieval and natural language processing. In order to cross the boundaries that exist between different languages, dictionaries are the most typical tools. However, the general‐purpose dictionary is less sensitive in both genre and domain. It is also impractical to manually construct tailored bilingual dictionaries or sophisticated multilingual thesauri for large applications. Corpus‐based approaches, which do not have the limitation of dictionaries, provide a statistical translation model with which to cross the language boundary. There are many domain‐specific parallel or comparable corpora that are employed in machine translation and cross‐lingual information retrieval. Most of these are corpora between Indo‐European languages, such as English/French and English/Spanish. The Asian/Indo‐European corpus, especially English/Chinese corpus, is relatively sparse. The objective of the present research is to construct English/Chinese parallel corpus automatically from the World Wide Web. In this paper, an alignment method is presented which is based on dynamic programming to identify the one‐to‐one Chinese and English title pairs. The method includes alignment at title level, word level and character level. The longest common subsequence (LCS) is applied to find the most reliable Chinese translation of an English word. As one word for a language may translate into two or more words repetitively in another language, the edit operation, deletion, is used to resolve redundancy. A score function is then proposed to determine the optimal title pairs. Experiments have been conducted to investigate the performance of the proposed method using the daily press release articles by the Hong Kong SAR government as the test bed. The precision of the result is 0.998 while the recall is 0.806. The release articles and speech articles, published by Hongkong & Shanghai Banking Corporation Limited, are also used to test our method, the precision is 1.00, and the recall is 0.948.

A Pattern Matching method for finding Noun and Proper Noun Translations from Noisy Parallel Corpora

Aligning Noisy Parallel Corpora Across Language Groups : Word Pair Feature Matching by Dynamic Time Warping

Noun Phrase Alignment in Chinese-English Bilingual Corpora

Transfer and Multi-Task Learning for Noun-Noun Compound Interpretation

Translation Algorithm of Proper Nouns in English-Chinese Translation Based on Lexical Knowledge Spectrum Map

Automatic construction of English/Chinese parallel corpora

Building Subject-aligned Comparable Corpora and Mining it for Truly Parallel Sentence Pairs

Extracting Paraphrase Patterns from Bilingual Parallel Corpora

Bilingual Text Extraction as Reading Comprehension

Bilingual lexicon induction from non-parallel corpora

The Construction of a Chinese-English Patent Parallel Corpus

Agreement-based Learning of Parallel Lexicons and Phrases from Non-Parallel Corpora

Noisy Parallel Data Alignment

Exploring transfer learning for Deep NLP systems on rarely annotated languages

Extracting Noun Phrases from Large-Scale Texts: A Hybrid Approach and Its Automatic Evaluation

Collocation Extraction Using Monolingual Word Alignment Method.

Aligning a Parallel English-Chinese Corpus Statistically with Lexical Criteria

Bilingual Corpus Mining and Multistage Fine-Tuning for Improving Machine Translation of Lecture Transcripts

Word Alignment by Fine-tuning Embeddings on Parallel Corpora

Noisy-parallel and comparable corpora filtering methodology for the extraction of bi-lingual equivalent data at sentence level

Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations