Abstract:Large language models (LLMs) exhibit outstanding performance in machine translation via in-context learning. In contrast to sentence-level translation, document-level translation (DOCMT) by LLMs based on in-context learning faces two major challenges: firstly, document translations generated by LLMs are often incoherent; secondly, the length of demonstration for in-context learning is usually limited. To address these issues, we propose a Context-Aware Prompting method (CAP), which enables LLMs to generate more accurate, cohesive, and coherent translations via in-context learning. CAP takes into account multi-level attention, selects the most relevant sentences to the current one as context, and then generates a summary from these collected sentences. Subsequently, sentences most similar to the summary are retrieved from the datastore as demonstrations, which effectively guide LLMs in generating cohesive and coherent translations. We conduct extensive experiments across various DOCMT tasks, and the results demonstrate the effectiveness of our approach, particularly in zero pronoun translation (ZPT) and literary translation tasks.

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve This paper aims to address two main challenges encountered when using large language models (LLMs) for document-level machine translation (DOCMT): 1. **Incoherent Document Translation**: Document translations generated through context learning often lack coherence. 2. **Limited Context Learning Example Length**: Due to the limited input window of LLMs, the length of examples used for context learning is restricted. To solve these issues, the authors propose a method called **Context-Aware Prompting (CAP)**, which selects relevant sentences as context through a multi-level attention mechanism and generates summaries to effectively guide LLMs in producing coherent and accurate document translations. Specifically, the CAP method includes the following three steps: 1. **Dynamic Context Window**: By combining word-level and sentence-level attention scores, select the sentences most relevant to the current sentence as context. 2. **Summarization and Retrieval**: Input the context into the LLM to generate a summary, then retrieve the parallel sentence pairs most similar to the summary from the data store. 3. **Inference**: Use the retrieved example pairs as demonstrations in the prompt to guide the LLM in generating document translations with a few examples. ### Experimental Validation To validate the effectiveness of the CAP method, the authors conducted extensive experiments on multiple document-level machine translation tasks, including zero pronoun translation (ZPT) and literary translation tasks. The experimental results show that the CAP method significantly improves translation quality across different scales of LLMs, particularly excelling in ZPT tasks and literary translation tasks. ### Main Contributions 1. **Exploration of Document-Level Translation**: Investigated the performance of LLMs in document-level translation, especially language phenomena such as zero pronoun translation. 2. **Proposed CAP Method**: Addressed the issues of translation coherence and context learning example length limitation through dynamic context windows and summarization techniques. 3. **Experimental Validation**: Conducted experiments on multiple document-level machine translation tasks, demonstrating that the CAP method significantly outperforms baseline methods, especially in ZPT tasks. ### Related Work - **Document-Level Machine Translation**: In recent years, many studies have proposed different DOCMT methods, including extending source-side context and document-to-document (Doc2Doc) translation. - **LLMs for Translation**: Although LLMs have shown outstanding performance in various NLP tasks, most studies focus on sentence-level translation, with less attention to document-level translation. Recent studies have shown that context learning can significantly improve the translation performance of LLMs. ### Conclusion Through the CAP method, the authors successfully addressed the issues of coherence and context learning example length limitation in document-level translation, providing a new solution for utilizing LLMs for high-quality document translation.

Efficiently Exploring Large Language Models for Document-Level Machine Translation with In-context Learning

Enhancing Document-level Translation of Large Language Model via Translation Mixed-instructions

Document-Level Machine Translation with Large Language Models

Adapting Large Language Models for Document-Level Machine Translation

Analyzing Context Utilization of LLMs in Document-Level Translation

Refining Translations with LLMs: A Constraint-Aware Iterative Prompting Approach

Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models

Adaptive Machine Translation with Large Language Models

A Novel Paradigm Boosting Translation Capabilities of Large Language Models

Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis

Towards Chapter-to-Chapter Context-Aware Literary Translation via Large Language Models

LLM×MapReduce: Simplified Long-Sequence Processing Using Large Language Models

Large language models effectively leverage document-level context for literary translation, but critical errors persist

Adapting LLMs for Efficient Context Processing through Soft Prompt Compression

Document-Level Language Models for Machine Translation

Exploring Human-Like Translation Strategy with Large Language Models

LLM$\times$MapReduce: Simplified Long-Sequence Processing using Large Language Models

DeMPT: Decoding-enhanced Multi-phase Prompt Tuning for Making LLMs Be Better Context-aware Translators

Large Language Models are In-Context Molecule Learners

Modeling Context With Linear Attention for Scalable Document-Level Translation

Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners