Abstract:Data selection has shown significant improvements in effective use of training data by extracting sentences from large general-domain corpora to adapt statistical machine translation (SMT) systems to in-domain data. This paper performs an in-depth analysis of three different sentence selection techniques. The first one is cosine tf-idf, which comes from the realm of information retrieval (IR). The second is perplexity-based approach, which can be found in the field of language modeling. These two data selection techniques applied to SMT have been already presented in the literature. However, edit distance for this task is proposed in this paper for the first time. After investigating the individual model, a combination of all three techniques is proposed at both corpus level and model level. Comparative experiments are conducted on Hong Kong law Chinese-English corpus and the results indicate the following: (i) the constraint degree of similarity measuring is not monotonically related to domain-specific translation quality; (ii) the individual selection models fail to perform effectively and robustly; but (iii) bilingual resources and combination methods are helpful to balance out-of-vocabulary (OOV) and irrelevant data; (iv) finally, our method achieves the goal to consistently boost the overall translation performance that can ensure optimal quality of a real-life SMT system.

Research on Translation Selection Based on Target Language Statistics

A Systematic Comparison of Data Selection Criteria for SMT Domain Adaptation

Research on the method of word sense disambiguation based on target language bigram

Improving Statistical Machine Translation with Selectional Preferences.

Graph-Based Collective Lexical Selection for Statistical Machine Translation.

Combining Multiple Chinese Word Segmentation Results for Statistical Machine Translation

Evaluation of All-words WSD for Chinese in Machine Translation

An Empirical Study on Development Set Selection Strategy for Machine Translation Learning.

Two statistics methods of Chinese word sense disambiguation

An empirical study on word segmentation for chinese machine translation

Chinese WSD Based on Selecting the Best Seeds from Collocations

WeTS: A Benchmark for Translation Suggestion

Improving Chinese to English SMT with Multiple CWS Results

A Graph-based Bilingual Corpus Selection Approach for SMT.

Back Deduction Based Testing for Word Sense Disambiguation Ability of Machine Translation Systems

Bilingual Recursive Neural Network Based Data Selection for Statistical Machine Translation

Icpe: A Hybrid Data Selection Model For Smt Domain Adaptation

A Context-Aware Topic Model for Statistical Machine Translation.

Example-based Machine Translation Based on Tree–string Correspondence and Statistical Generation

Topic Model Based Adaptation Data Selection for Domain-Specific Machine Translation.

Enhancing Statistical Machine Translation with Character Alignment