Abstract:It is well-known that document context is vital for resolving a range of translation ambiguities, and in fact the document setting is the most natural setting for nearly all translation. It is therefore unfortunate that machine translation -- both research and production -- largely remains stuck in a decades-old sentence-level translation paradigm. It is also an increasingly glaring problem in light of competitive pressure from large language models, which are natively document-based. Much work in document-context machine translation exists, but for various reasons has been unable to catch hold. This paper suggests a path out of this rut by addressing three impediments at once: what architectures should we use? where do we get document-level information for training them? and how do we know whether they are any good? In contrast to work on specialized architectures, we show that the standard Transformer architecture is sufficient, provided it has enough capacity. Next, we address the training data issue by taking document samples from back-translated data only, where the data is not only more readily available, but is also of higher quality compared to parallel document data, which may contain machine translation output. Finally, we propose generative variants of existing contrastive metrics that are better able to discriminate among document systems. Results in four large-data language pairs (DE$\rightarrow$EN, EN$\rightarrow$DE, EN$\rightarrow$FR, and EN$\rightarrow$RU) establish the success of these three pieces together in improving document-level performance.

A Comparison of Approaches to Document-level Machine Translation

Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation

Document-level Consistency Verification in Machine Translation.

A Survey on Document-level Neural Machine Translation: Methods and Evaluation

Document-Level Machine Translation Evaluation with Gist Consistency and Text Cohesion.

Document-Level Machine Translation Evaluation Metrics Enhanced with Simplified Lexical Chain.

Document-Level Language Models for Machine Translation

Non-Autoregressive Document-Level Machine Translation

Rethinking Document-level Neural Machine Translation

A Document-Level Machine Translation Quality Estimation Model Based on Centering Theory

Escaping the sentence-level paradigm in machine translation

On Search Strategies for Document-Level Neural Machine Translation

The Comparison of Translationese in Machine Translation and Human Transation in terms of Translation Relations

A Review of Discourse-level Machine Translation

Document-Level Machine Translation with Large Language Models

Research on Machine Translation Automatic Evaluation Based on Extended Reference

Comparison between Human Translation and Machine Translation in Translating the Publicity Text of Haihunhou Museum

Cache-based Document-Level Statistical Machine Translation

Statistical machine translation based on LDA

Difficulty-Aware Machine Translation Evaluation

Exploring Discourse Structure in Document-level Machine Translation