Abstract:Monolingual data have been demonstrated to be helpful in improving translation quality of both statistical machine translation (SMT) systems and neural machine translation (NMT) systems, especially in resource-poor or domain adaptation tasks where parallel data are not rich enough. In this paper, we propose a novel approach to better leveraging monolingual data for neural machine translation by jointly learning source-to-target and target-to-source NMT models for a language pair with a joint EM optimization method. The training process starts with two initial NMT models pre-trained on parallel data for each direction, and these two models are iteratively updated by incrementally decreasing translation losses on training data.In each iteration step, both NMT models are first used to translate monolingual data from one language to the other, forming pseudo-training data of the other NMT model. Then two new NMT models are learnt from parallel data together with the pseudo training data. Both NMT models are expected to be improved and better pseudo-training data can be generated in next step. Experiment results on Chinese-English and English-German translation tasks show that our approach can simultaneously improve translation quality of source-to-target and target-to-source models, significantly outperforming strong baseline systems which are enhanced with monolingual data for model training including back-translation.

Joint Alignment and Artificial Data Generation: an Empirical Study of Pivot-based Machine Transliteration

Joint Training for Pivot-based Neural Machine Translation.

Improving Pivot-Based Statistical Machine Translation Using Random Walk.

Neural Machine Translation with Pivot Languages

Improving Pivot-Based Statistical Machine Translation by Pivoting the Co-occurrence Count of Phrase Pairs

3 Joint Training for Pivot-based NMT 3 . 1 Training Objective

Improving low-resource machine transliteration by using 3-way transfer learning

Topic-aware pivot language approach for statisticalmachine translation

How Transliterations Improve Crosslingual Alignment

Revisiting Pivot Language Approach for Machine Translation.

End-to-end Training and Decoding for Pivot-based Cascaded Translation Model

PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning

Combining Multiple Alignments to Improve Machine Translation.

Towards Integrated Machine Translation Using Structural Alignment From Syntax-Augmented Synchronous Parsing

Weighted Alignment Matrices for Statistical Machine Translation.

Pivot Language Approach for Phrase-Based Statistical Machine Translation

Jointly Learning to Align and Translate with Transformer Models

Investigating Multi-Pivot Ensembling with Massively Multilingual Machine Translation Models

Joint Training for Neural Machine Translation Models with Monolingual Data

Improving Joint Speech-Text Representations Without Alignment

When Is TTS Augmentation Through a Pivot Language Useful?