Abstract:Monolingual data have been demonstrated to be helpful in improving translation quality of both statistical machine translation (SMT) systems and neural machine translation (NMT) systems, especially in resource-poor or domain adaptation tasks where parallel data are not rich enough. In this paper, we propose a novel approach to better leveraging monolingual data for neural machine translation by jointly learning source-to-target and target-to-source NMT models for a language pair with a joint EM optimization method. The training process starts with two initial NMT models pre-trained on parallel data for each direction, and these two models are iteratively updated by incrementally decreasing translation losses on training data.In each iteration step, both NMT models are first used to translate monolingual data from one language to the other, forming pseudo-training data of the other NMT model. Then two new NMT models are learnt from parallel data together with the pseudo training data. Both NMT models are expected to be improved and better pseudo-training data can be generated in next step. Experiment results on Chinese-English and English-German translation tasks show that our approach can simultaneously improve translation quality of source-to-target and target-to-source models, significantly outperforming strong baseline systems which are enhanced with monolingual data for model training including back-translation.

Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages

Neural Machine Translation with Pivot Languages

Simultaneous Multi-Pivot Neural Machine Translation

Joint Training for Pivot-based Neural Machine Translation.

3 Joint Training for Pivot-based NMT 3 . 1 Training Objective

Unsupervised Pivot Translation for Distant Languages

Improving low-resource machine transliteration by using 3-way transfer learning

End-to-end Training and Decoding for Pivot-based Cascaded Translation Model

Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies

Revisiting Pivot Language Approach for Machine Translation.

On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translation

Improving Neural Machine Translation by Bidirectional Training

Improving Language Transfer Capability of Decoder-only Architecture in Multilingual Neural Machine Translation

A Study of Multilingual Neural Machine Translation

Topic-aware pivot language approach for statisticalmachine translation

Improving Multilingual Translation by Representation and Gradient Regularization

Pivot Language Approach for Phrase-Based Statistical Machine Translation

Joint Training for Neural Machine Translation Models with Monolingual Data

Improving Pivot-Based Statistical Machine Translation by Pivoting the Co-occurrence Count of Phrase Pairs

Improving Pivot-Based Statistical Machine Translation Using Random Walk.

Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation