Abstract:Unsupervised neural machine translation (UNMT) has achieved remarkable results for several language pairs, such as French–English and German–English. Most previous studies have focused on modeling UNMT systems; few studies have investigated the effect of UNMT on specific languages. In this article, we first empirically investigate UNMT for four diverse language pairs (French/German/Chinese/Japanese–English). We confirm that the performance of UNMT in translation tasks for similar language pairs (French/German–English) is dramatically better than for distant language pairs (Chinese/Japanese–English). We empirically show that the lack of shared words and different word orderings are the main reasons that lead UNMT to underperform in Chinese/Japanese–English. Based on these findings, we propose several methods, including artificial shared words and pre-ordering, to improve the performance of UNMT for distant language pairs. Moreover, we propose a simple general method to improve translation performance for all these four language pairs. The existing UNMT model can generate a translation of a reasonable quality after a few training epochs owing to a denoising mechanism and shared latent representations. However, learning shared latent representations restricts the performance of translation in both directions, particularly for distant language pairs, while denoising dramatically delays convergence by continuously modifying the training data. To avoid these problems, we propose a simple, yet effective and efficient, approach that (like UNMT) relies solely on monolingual corpora: pseudo-data-based unsupervised neural machine translation. Experimental results for these four language pairs show that our proposed methods significantly outperform UNMT baselines.

Unsupervised Neural Machine Translation with Cross-Lingual Language Representation Agreement

Unsupervised Bilingual Word Embedding Agreement for Unsupervised Neural Machine Translation

Unsupervised Neural Machine Translation for Similar and Distant Language Pairs: an Empirical Study

Advances and Challenges in Unsupervised Neural Machine Translation.

Cross-lingual Supervision Improves Unsupervised Neural Machine Translation

Language Model-Driven Unsupervised Neural Machine Translation

Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation

Unified Model Learning for Various Neural Machine Translation

Demystifying Learning of Unsupervised Neural Machine Translation

Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios

Multilingual Agreement for Multilingual Neural Machine Translation

Unsupervised Neural Machine Translation with Indirect Supervision.

Explicit Cross-lingual Pre-training for Unsupervised Machine Translation

A2R2: Robust Unsupervised Neural Machine Translation with Adversarial Attack and Regularization on Representations

Unsupervised Neural Machine Translation with SMT As Posterior Regularization

Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation

Reference Language based Unsupervised Neural Machine Translation

An Empirical study of Unsupervised Neural Machine Translation: analyzing NMT output, model's behavior and sentences' contribution

Phrase-Based & Neural Unsupervised Machine Translation

Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training

Mongolian-Chinese Unsupervised Neural Machine Translation With Lexical Feature