Six Challenges for Neural Machine Translation

Philipp Koehn,Rebecca Knowles

DOI: https://doi.org/10.48550/arXiv.1706.03872

2017-06-12

Computation and Language

Abstract:We explore six challenges for neural machine translation: domain mismatch, amount of training data, rare words, long sentences, word alignment, and beam search. We show both deficiencies and improvements over the quality of phrase-based statistical machine translation.

What problem does this paper attempt to address?

Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models

Jianhui Pang,Fanghua Ye,Longyue Wang,Dian Yu,Derek F. Wong,Shuming Shi,Zhaopeng Tu

2024-01-17

Abstract:The evolution of Neural Machine Translation (NMT) has been significantly influenced by six core challenges (Koehn and Knowles, 2017), which have acted as benchmarks for progress in this field. This study revisits these challenges, offering insights into their ongoing relevance in the context of advanced Large Language Models (LLMs): domain mismatch, amount of parallel data, rare word prediction, translation of long sentences, attention model as word alignment, and sub-optimal beam search. Our empirical findings indicate that LLMs effectively lessen the reliance on parallel data for major languages in the pretraining phase. Additionally, the LLM-based translation system significantly enhances the translation of long sentences that contain approximately 80 words and shows the capability to translate documents of up to 512 words. However, despite these significant improvements, the challenges of domain mismatch and prediction of rare words persist. While the challenges of word alignment and beam search, specifically associated with NMT, may not apply to LLMs, we identify three new challenges for LLMs in translation tasks: inference efficiency, translation of low-resource languages in the pretraining phase, and human-aligned evaluation. The datasets and models are released at

Computation and Language
Challenges of Neural Machine Translation for Short Texts

Yu Wan,Baosong Yang,Derek Fai Wong,Lidia Sam Chao,Liang Yao,Haibo Zhang,Boxing Chen

DOI: https://doi.org/10.1162/coli_a_00435

IF: 7.778

2022-03-07

Computational Linguistics

Abstract:Abstract Short texts (STs) present in a variety of scenarios, including query, dialog, and entity names. Most of the exciting studies in neural machine translation (NMT) are wild about tackling open problems concerning long sentences rather than short ones. The intuition behind is that, with respect to human learning and processing, short sequences are generally regarded as easy examples. In this paper, we first dispel this speculation via conducting preliminary experiments, showing that the conventional state-of-the-art NMT approach, i.e. Transformer (Vaswani et al. 2017), still severely suffers from over-translation and mistranslation errors over STs. After empirically investigating the rationale behind, we summarized two challenges in NMT for STs associated with translation error types above, respectively: 1) the imbalanced length distribution in training set intensifies model inference calibration over STs, leading to more over-translation cases on STs; 2) the lack of contextual information makes NMT have higher data uncertainty on short sentences, and NMT model is troubled by considerable mistranslation errors. Some existing approaches, like balancing data distribution for training (e.g. data upsampling) and complementing contextual information (e.g. introducing translation memory) can pertinently alleviate the translation issues in NMT for STs. We are delighted to embrace researchers investigate other challenges in NMT for STs, thus reducing ST translation errors and enhancing translation quality.

computer science, artificial intelligence, interdisciplinary applications,linguistics
On the Integration of LinguisticFeatures into Statistical and Neural Machine Translation

Eva Vanmassenhove

DOI: https://doi.org/10.48550/arXiv.2003.14324

2020-03-31

Computation and Language

Abstract:New machine translations (MT) technologies are emerging rapidly and with them, bold claims of achieving human parity such as: (i) the results produced approach "accuracy achieved by average bilingual human translators" (Wu et al., 2017b) or (ii) the "translation quality is at human parity when compared to professional human translators" (Hassan et al., 2018) have seen the light of day (Laubli et al., 2018). Aside from the fact that many of these papers craft their own definition of human parity, these sensational claims are often not supported by a complete analysis of all aspects involved in translation. Establishing the discrepancies between the strengths of statistical approaches to MT and the way humans translate has been the starting point of our research. By looking at MT output and linguistic theory, we were able to identify some remaining issues. The problems range from simple number and gender agreement errors to more complex phenomena such as the correct translation of aspectual values and tenses. Our experiments confirm, along with other studies (Bentivogli et al., 2016), that neural MT has surpassed statistical MT in many aspects. However, some problems remain and others have emerged. We cover a series of problems related to the integration of specific linguistic features into statistical and neural MT, aiming to analyse and provide a solution to some of them. Our work focuses on addressing three main research questions that revolve around the complex relationship between linguistics and MT in general. We identify linguistic information that is lacking in order for automatic translation systems to produce more accurate translations and integrate additional features into the existing pipelines. We identify overgeneralization or 'algorithmic bias' as a potential drawback of neural MT and link it to many of the remaining linguistic issues.
Neural machine translation: Challenges, progress and future

JiaJun Zhang,ChengQing Zong

DOI: https://doi.org/10.1007/s11431-020-1632-x

2020-09-15

Science China Technological Sciences

Abstract:Machine translation (MT) is a technique that leverages computers to translate human languages automatically. Nowadays, neural machine translation (NMT) which models direct mapping between source and target languages with deep neural networks has achieved a big breakthrough in translation performance and become the de facto paradigm of MT. This article makes a review of NMT framework, discusses the challenges in NMT, introduces some exciting recent progresses and finally looks forward to some potential future research trends.

materials science, multidisciplinary,engineering
Domain Robustness in Neural Machine Translation

Mathias Müller,Annette Rios,Rico Sennrich

DOI: https://doi.org/10.48550/arXiv.1911.03109

2020-09-24

Abstract:Translating text that diverges from the training domain is a key challenge for machine translation. Domain robustness---the generalization of models to unseen test domains---is low for both statistical (SMT) and neural machine translation (NMT). In this paper, we study the performance of SMT and NMT models on out-of-domain test sets. We find that in unknown domains, SMT and NMT suffer from very different problems: SMT systems are mostly adequate but not fluent, while NMT systems are mostly fluent, but not adequate. For NMT, we identify such hallucinations (translations that are fluent but unrelated to the source) as a key reason for low domain robustness. To mitigate this problem, we empirically compare methods that are reported to improve adequacy or in-domain robustness in terms of their effectiveness at improving domain robustness. In experiments on German to English OPUS data, and German to Romansh (a low-resource setting) we find that several methods improve domain robustness. While those methods do lead to higher BLEU scores overall, they only slightly increase the adequacy of translations compared to SMT.

Computation and Language
Is Neural Machine Translation Ready for Deployment? A Case Study on 30 Translation Directions

Marcin Junczys-Dowmunt,Tomasz Dwojak,Hieu Hoang

DOI: https://doi.org/10.48550/arXiv.1610.01108

2016-12-01

Abstract:In this paper we provide the largest published comparison of translation quality for phrase-based SMT and neural machine translation across 30 translation directions. For ten directions we also include hierarchical phrase-based MT. Experiments are performed for the recently published United Nations Parallel Corpus v1.0 and its large six-way sentence-aligned subcorpus. In the second part of the paper we investigate aspects of translation speed, introducing AmuNMT, our efficient neural machine translation decoder. We demonstrate that current neural machine translation could already be used for in-production systems when comparing words-per-second ratios.

Computation and Language
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

Melvin Johnson,Mike Schuster,Quoc V. Le,Maxim Krikun,Yonghui Wu,Zhifeng Chen,Nikhil Thorat,Fernanda Viégas,Martin Wattenberg,Greg Corrado,Macduff Hughes,Jeffrey Dean,Melvin Johnson,Mike Schuster,Quoc V. Le,Maxim Krikun,Yonghui Wu,Zhifeng Chen,Nikhil Thorat,Fernanda Viégas,Martin Wattenberg,Greg Corrado,Macduff Hughes,Jeffrey Dean

DOI: https://doi.org/10.1162/tacl_a_00065

IF: 10.9

2017-12-01

Transactions of the Association for Computational Linguistics

Abstract:We propose a simple solution to use a single Neural Machine Translation (NMT) model to translate between multiple languages. Our solution requires no changes to the model architecture from a standard NMT system but instead introduces an artificial token at the beginning of the input sentence to specify the required target language. Using a shared wordpiece vocabulary, our approach enables Multilingual NMT systems using a single model. On the WMT’14 benchmarks, a single multilingual model achieves comparable performance for English→French and surpasses state-of-theart results for English→German. Similarly, a single multilingual model surpasses state-of-the-art results for French→English and German→English on WMT’14 and WMT’15 benchmarks, respectively. On production corpora, multilingual models of up to twelve language pairs allow for better translation of many individual pairs. Our models can also learn to perform implicit bridging between language pairs never seen explicitly during training, showing that transfer learning and zero-shot translation is possible for neural translation. Finally, we show analyses that hints at a universal interlingua representation in our models and also show some interesting examples when mixing languages.
Machine Translation: Phrase-Based, Rule-Based and Neural Approaches with Linguistic Evaluation

Vivien Macketanz,Eleftherios Avramidis,Aljoscha Burchardt,Jindrich Helcl,Ankit Srivastava

DOI: https://doi.org/10.1515/cait-2017-0014

2017-06-01

Cybernetics and Information Technologies

Abstract:Abstract In this article we present a novel linguistically driven evaluation method and apply it to the main approaches of Machine Translation (Rule-based, Phrase-based, Neural) to gain insights into their strengths and weaknesses in much more detail than provided by current evaluation schemes. Translating between two languages requires substantial modelling of knowledge about the two languages, about translation, and about the world. Using English-German IT-domain translation as a case-study, we also enhance the Phrase-based system by exploiting parallel treebanks for syntax-aware phrase extraction and by interfacing with Linked Open Data (LOD) for extracting named entity translations in a post decoding framework.
Assessing the Tolerance of Neural Machine Translation Systems Against Speech Recognition Errors

Nicholas Ruiz,Mattia Antonino Di Gangi,Nicola Bertoldi,Marcello Federico

DOI: https://doi.org/10.48550/arXiv.1904.10997

2019-04-24

Computation and Language

Abstract:Machine translation systems are conventionally trained on textual resources that do not model phenomena that occur in spoken language. While the evaluation of neural machine translation systems on textual inputs is actively researched in the literature , little has been discovered about the complexities of translating spoken language data with neural models. We introduce and motivate interesting problems one faces when considering the translation of automatic speech recognition (ASR) outputs on neural machine translation (NMT) systems. We test the robustness of sentence encoding approaches for NMT encoder-decoder modeling, focusing on word-based over byte-pair encoding. We compare the translation of utterances containing ASR errors in state-of-the-art NMT encoder-decoder systems against a strong phrase-based machine translation baseline in order to better understand which phenomena present in ASR outputs are better represented under the NMT framework than approaches that represent translation as a linear model.
Domain, Translationese and Noise in Synthetic Data for Neural Machine Translation

Nikolay Bogoychev,Rico Sennrich

DOI: https://doi.org/10.48550/arXiv.1911.03362

2020-10-04

Abstract:The quality of neural machine translation can be improved by leveraging additional monolingual resources to create synthetic training data. Source-side monolingual data can be (forward-)translated into the target language for self-training; target-side monolingual data can be back-translated. It has been widely reported that back-translation delivers superior results, but could this be due to artefacts in the test sets? We perform a case study using French-English news translation task and separate test sets based on their original languages. We show that forward translation delivers superior gains in terms of BLEU on sentences that were originally in the source language, complementing previous studies which show large improvements with back-translation on sentences that were originally in the target language. To better understand when and why forward and back-translation are effective, we study the role of domains, translationese, and noise. While translationese effects are well known to influence MT evaluation, we also find evidence that news data from different languages shows subtle domain differences, which is another explanation for varying performance on different portions of the test set. We perform additional low-resource experiments which demonstrate that forward translation is more sensitive to the quality of the initial translation system than back-translation, and tends to perform worse in low-resource settings.

Computation and Language,Machine Learning
Phrase-Based & Neural Unsupervised Machine Translation

Guillaume Lample,Myle Ott,Alexis Conneau,Ludovic Denoyer,Marc'Aurelio Ranzato

DOI: https://doi.org/10.48550/arXiv.1804.07755

2018-04-20

Computation and Language

Abstract:Machine translation systems achieve near human-level performance on some languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences, which hinders their applicability to the majority of language pairs. This work investigates how to learn to translate when having access to only large monolingual corpora in each language. We propose two model variants, a neural and a phrase-based model. Both versions leverage a careful initialization of the parameters, the denoising effect of language models and automatic generation of parallel data by iterative back-translation. These models are significantly better than methods from the literature, while being simpler and having fewer hyper-parameters. On the widely used WMT'14 English-French and WMT'16 German-English benchmarks, our models respectively obtain 28.1 and 25.2 BLEU points without using a single parallel sentence, outperforming the state of the art by more than 11 BLEU points. On low-resource languages like English-Urdu and English-Romanian, our methods achieve even better results than semi-supervised and supervised approaches leveraging the paucity of available bitexts. Our code for NMT and PBSMT is publicly available.
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Yonghui Wu,Mike Schuster,Zhifeng Chen,Quoc V. Le,Mohammad Norouzi,Wolfgang Macherey,Maxim Krikun,Yuan Cao,Qin Gao,Klaus Macherey,Jeff Klingner,Apurva Shah,Melvin Johnson,Xiaobing Liu,Łukasz Kaiser,Stephan Gouws,Yoshikiyo Kato,Taku Kudo,Hideto Kazawa,Keith Stevens,George Kurian,Nishant Patil,Wei Wang,Cliff Young,Jason Smith,Jason Riesa,Alex Rudnick,Oriol Vinyals,Greg Corrado,Macduff Hughes,Jeffrey Dean

DOI: https://doi.org/10.48550/arXiv.1609.08144

2016-09-26

Computation and Language

Abstract:Neural Machine Translation (NMT) is an end-to-end learning approach for automated translation, with the potential to overcome many of the weaknesses of conventional phrase-based translation systems. Unfortunately, NMT systems are known to be computationally expensive both in training and in translation inference. Also, most NMT systems have difficulty with rare words. These issues have hindered NMT's use in practical deployments and services, where both accuracy and speed are essential. In this work, we present GNMT, Google's Neural Machine Translation system, which attempts to address many of these issues. Our model consists of a deep LSTM network with 8 encoder and 8 decoder layers using attention and residual connections. To improve parallelism and therefore decrease training time, our attention mechanism connects the bottom layer of the decoder to the top layer of the encoder. To accelerate the final translation speed, we employ low-precision arithmetic during inference computations. To improve handling of rare words, we divide words into a limited set of common sub-word units ("wordpieces") for both input and output. This method provides a good balance between the flexibility of "character"-delimited models and the efficiency of "word"-delimited models, naturally handles translation of rare words, and ultimately improves the overall accuracy of the system. Our beam search technique employs a length-normalization procedure and uses a coverage penalty, which encourages generation of an output sentence that is most likely to cover all the words in the source sentence. On the WMT'14 English-to-French and English-to-German benchmarks, GNMT achieves competitive results to state-of-the-art. Using a human side-by-side evaluation on a set of isolated simple sentences, it reduces translation errors by an average of 60% compared to Google's phrase-based production system.
Neural Machine Translation: A Review and Survey

Felix Stahlberg

DOI: https://doi.org/10.48550/arXiv.1912.02047

2019-12-04

Computation and Language

Abstract:The field of machine translation (MT), the automatic translation of written text from one natural language into another, has experienced a major paradigm shift in recent years. Statistical MT, which mainly relies on various count-based models and which used to dominate MT research for decades, has largely been superseded by neural machine translation (NMT), which tackles translation with a single neural network. In this work we will trace back the origins of modern NMT architectures to word and sentence embeddings and earlier examples of the encoder-decoder network family. We will conclude with a survey of recent trends in the field.
The Reality of Multi-Lingual Machine Translation

Tom Kocmi,Dominik Macháček,Ondřej Bojar

DOI: https://doi.org/10.48550/arXiv.2202.12814

2022-02-25

Computation and Language

Abstract:Our book "The Reality of Multi-Lingual Machine Translation" discusses the benefits and perils of using more than two languages in machine translation systems. While focused on the particular task of sequence-to-sequence processing and multi-task learning, the book targets somewhat beyond the area of natural language processing. Machine translation is for us a prime example of deep learning applications where human skills and learning capabilities are taken as a benchmark that many try to match and surpass. We document that some of the gains observed in multi-lingual translation may result from simpler effects than the assumed cross-lingual transfer of knowledge. In the first, rather general part, the book will lead you through the motivation for multi-linguality, the versatility of deep neural networks especially in sequence-to-sequence tasks to complications of this learning. We conclude the general part with warnings against too optimistic and unjustified explanations of the gains that neural networks demonstrate. In the second part, we fully delve into multi-lingual models, with a particularly careful examination of transfer learning as one of the more straightforward approaches utilizing additional languages. The recent multi-lingual techniques, including massive models, are surveyed and practical aspects of deploying systems for many languages are discussed. The conclusion highlights the open problem of machine understanding and reminds of two ethical aspects of building large-scale models: the inclusivity of research and its ecological trace.
Progress in Machine Translation

Haifeng Wang,Hua Wu,Zhongjun He,Liang Huang,Kenneth Ward Church

DOI: https://doi.org/10.1016/j.eng.2021.03.023

IF: 12.834

2021-01-01

Engineering

Abstract:After more than 70 years of evolution, great achievements have been made in machine translation. Especially in recent years, translation quality has been greatly improved with the emergence of neural machine translation (NMT). In this article, we first review the history of machine translation from rule-based machine translation to example-based machine translation and statistical machine translation. We then introduce NMT in more detail, including the basic framework and the current dominant framework, Transformer, as well as multilingual translation models to deal with the data sparseness problem. In addition, we introduce cutting-edge simultaneous translation methods that achieve a balance between translation quality and latency. We then describe various products and applications of machine translation. At the end of this article, we briefly discuss challenges and future research directions in this field.
Scaling neural machine translation to 200 languages

NLLB Team,Marta R. Costa-jussà,James Cross,Onur Çelebi,Maha Elbayad,Kenneth Heafield,Kevin Heffernan,Elahe Kalbassi,Janice Lam,Daniel Licht,Jean Maillard,Anna Sun,Skyler Wang,Guillaume Wenzek,Al Youngblood,Bapi Akula,Loic Barrault,Gabriel Mejia Gonzalez,Prangthip Hansanti,John Hoffman,Semarley Jarrett,Kaushik Ram Sadagopan,Dirk Rowe,Shannon Spruit,Chau Tran,Pierre Andrews,Necip Fazil Ayan,Shruti Bhosale,Sergey Edunov,Angela Fan,Cynthia Gao,Vedanuj Goswami,Francisco Guzmán,Philipp Koehn,Alexandre Mourachko,Christophe Ropers,Safiyyah Saleem,Holger Schwenk,Jeff Wang

DOI: https://doi.org/10.1038/s41586-024-07335-x

IF: 64.8

2024-06-06

Nature

Abstract:The development of neural techniques has opened up new avenues for research in machine translation. Today, neural machine translation (NMT) systems can leverage highly multilingual capacities and even perform zero-shot translation, delivering promising results in terms of language coverage and quality. However, scaling quality NMT requires large volumes of parallel bilingual data, which are not equally available for the 7,000+ languages in the world 1 . Focusing on improving the translation qualities of a relatively small group of high-resource languages comes at the expense of directing research attention to low-resource languages, exacerbating digital inequities in the long run. To break this pattern, here we introduce No Language Left Behind—a single massively multilingual model that leverages transfer learning across languages. We developed a conditional computational model based on the Sparsely Gated Mixture of Experts architecture 2,3,4,5,6,7 , which we trained on data obtained with new mining techniques tailored for low-resource languages. Furthermore, we devised multiple architectural and training improvements to counteract overfitting while training on thousands of tasks. We evaluated the performance of our model over 40,000 translation directions using tools created specifically for this purpose—an automatic benchmark (FLORES-200), a human evaluation metric (XSTS) and a toxicity detector that covers every language in our model. Compared with the previous state-of-the-art models, our model achieves an average of 44% improvement in translation quality as measured by BLEU. By demonstrating how to scale NMT to 200 languages and making all contributions in this effort freely available for non-commercial use, our work lays important groundwork for the development of a universal translation system.

multidisciplinary sciences
Towards Neural Machine Translation with Partially Aligned Corpora

Yining Wang,Yang Zhao,Jiajun Zhang,Chengqing Zong,Zhengshan Xue

DOI: https://doi.org/10.48550/arXiv.1711.01006

2017-11-03

Abstract:While neural machine translation (NMT) has become the new paradigm, the parameter optimization requires large-scale parallel data which is scarce in many domains and language pairs. In this paper, we address a new translation scenario in which there only exists monolingual corpora and phrase pairs. We propose a new method towards translation with partially aligned sentence pairs which are derived from the phrase pairs and monolingual corpora. To make full use of the partially aligned corpora, we adapt the conventional NMT training method in two aspects. On one hand, different generation strategies are designed for aligned and unaligned target words. On the other hand, a different objective function is designed to model the partially aligned parts. The experiments demonstrate that our method can achieve a relatively good result in such a translation scenario, and tiny bitexts can boost translation quality to a large extent.

Computation and Language
Evaluating Discourse Phenomena in Neural Machine Translation

Rachel Bawden,Rico Sennrich,Alexandra Birch,Barry Haddow

DOI: https://doi.org/10.48550/arXiv.1711.00513

2018-04-20

Abstract:For machine translation to tackle discourse phenomena, models must have access to extra-sentential linguistic context. There has been recent interest in modelling context in neural machine translation (NMT), but models have been principally evaluated with standard automatic metrics, poorly adapted to evaluating discourse phenomena. In this article, we present hand-crafted, discourse test sets, designed to test the models' ability to exploit previous source and target sentences. We investigate the performance of recently proposed multi-encoder NMT models trained on subtitles for English to French. We also explore a novel way of exploiting context from the previous sentence. Despite gains using BLEU, multi-encoder models give limited improvement in the handling of discourse phenomena: 50% accuracy on our coreference test set and 53.5% for coherence/cohesion (compared to a non-contextual baseline of 50%). A simple strategy of decoding the concatenation of the previous and current sentence leads to good performance, and our novel strategy of multi-encoding and decoding of two sentences leads to the best performance (72.5% for coreference and 57% for coherence/cohesion), highlighting the importance of target-side context.

Computation and Language
Neural Machine Translation with Constraints

Deyi XIONG,Junhui LI,Xing WANG,Biao ZHANG

DOI: https://doi.org/10.1360/n112017-00222

2018-01-01

Abstract:Neural machine translation (NMT), powered by deep learning, is an emerging machine translation paradigm that has been advancing rapidly in recent years. It has become mainstream technology in both academia and industry of machine translation. This paper provides an overview of our research work on NMT. It particularly focuses on a series of NMT models proposed for considering a variety of useful information and knowledge constraints, which include variational NMT with constraints of latent variables, NMT advised by statistical machine translation, and NMT with syntactical constraints from the source language. In addition to this overview, this paper presents an outlook of the future trends in NMT.
A Reinforced Generation of Adversarial Examples for Neural Machine Translation

Wei Zou,Shujian Huang,Jun Xie,Xinyu Dai,Jiajun Chen

DOI: https://doi.org/10.18653/v1/2020.acl-main.319

2020-01-01

Abstract:Neural machine translation systems tend to fail on less decent inputs despite its significant efficacy, which may significantly harm the credibility of these systems—fathoming how and when neural-based systems fail in such cases is critical for industrial maintenance. Instead of collecting and analyzing bad cases using limited handcrafted error features, here we investigate this issue by generating adversarial examples via a new paradigm based on reinforcement learning. Our paradigm could expose pitfalls for a given performance metric, e.g., BLEU, and could target any given neural machine translation architecture. We conduct experiments of adversarial attacks on two mainstream neural machine translation architectures, RNN-search, and Transformer. The results show that our method efficiently produces stable attacks with meaning-preserving adversarial examples. We also present a qualitative and quantitative analysis for the preference pattern of the attack, demonstrating its capability of pitfall exposure.

Six Challenges for Neural Machine Translation

Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models

Challenges of Neural Machine Translation for Short Texts

On the Integration of LinguisticFeatures into Statistical and Neural Machine Translation

Neural machine translation: Challenges, progress and future

Domain Robustness in Neural Machine Translation

Is Neural Machine Translation Ready for Deployment? A Case Study on 30 Translation Directions

Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

Machine Translation: Phrase-Based, Rule-Based and Neural Approaches with Linguistic Evaluation

Assessing the Tolerance of Neural Machine Translation Systems Against Speech Recognition Errors

Domain, Translationese and Noise in Synthetic Data for Neural Machine Translation

Phrase-Based & Neural Unsupervised Machine Translation

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Neural Machine Translation: A Review and Survey

The Reality of Multi-Lingual Machine Translation

Progress in Machine Translation

Scaling neural machine translation to 200 languages

Towards Neural Machine Translation with Partially Aligned Corpora

Evaluating Discourse Phenomena in Neural Machine Translation

Neural Machine Translation with Constraints

A Reinforced Generation of Adversarial Examples for Neural Machine Translation