Abstract:Unsupervised neural machine translation (UNMT) has achieved remarkable results for several language pairs, such as French–English and German–English. Most previous studies have focused on modeling UNMT systems; few studies have investigated the effect of UNMT on specific languages. In this article, we first empirically investigate UNMT for four diverse language pairs (French/German/Chinese/Japanese–English). We confirm that the performance of UNMT in translation tasks for similar language pairs (French/German–English) is dramatically better than for distant language pairs (Chinese/Japanese–English). We empirically show that the lack of shared words and different word orderings are the main reasons that lead UNMT to underperform in Chinese/Japanese–English. Based on these findings, we propose several methods, including artificial shared words and pre-ordering, to improve the performance of UNMT for distant language pairs. Moreover, we propose a simple general method to improve translation performance for all these four language pairs. The existing UNMT model can generate a translation of a reasonable quality after a few training epochs owing to a denoising mechanism and shared latent representations. However, learning shared latent representations restricts the performance of translation in both directions, particularly for distant language pairs, while denoising dramatically delays convergence by continuously modifying the training data. To avoid these problems, we propose a simple, yet effective and efficient, approach that (like UNMT) relies solely on monolingual corpora: pseudo-data-based unsupervised neural machine translation. Experimental results for these four language pairs show that our proposed methods significantly outperform UNMT baselines.

Semi-Supervised Neural Machine Translation Via Marginal Distribution Estimation

Semi-supervised neural machine translation via marginal distribution estimation

Dual Transfer Learning for Neural Machine Translation with Marginal Distribution Regularization

Semi-Supervised Learning for Neural Machine Translation

Exploiting monolingual data at scale for neural machine translation

Exploiting Monolingual Data at Scale for Neural Machine Translation.

Unsupervised Neural Machine Translation with SMT As Posterior Regularization

Language Model-Driven Unsupervised Neural Machine Translation

Bridging the Data Gap Between Training and Inference for Unsupervised Neural Machine Translation

Phrase-Based & Neural Unsupervised Machine Translation

Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation

Unsupervised Neural Machine Translation with Indirect Supervision.

Towards Neural Machine Translation with Partially Aligned Corpora

Unsupervised Neural Machine Translation with Future Rewarding.

Neural Machine Translation Advised by Statistical Machine Translation

Unpaired Multimodal Neural Machine Translation via Reinforcement Learning

Improving Multilingual Translation by Representation and Gradient Regularization

Unsupervised Neural Machine Translation for Similar and Distant Language Pairs: an Empirical Study

Joint Training for Neural Machine Translation Models with Monolingual Data

Maximum Expected Likelihood Estimation for Zero-resource Neural Machine Translation.