Abstract:Consistency is a key requirement of high-quality translation. It is especially important to adhere to pre-approved terminology and adapt to corrected translations in domain-specific projects. Machine translation (MT) has achieved significant progress in the area of domain adaptation. However, real-time adaptation remains challenging. Large-scale language models (LLMs) have recently shown interesting capabilities of in-context learning, where they learn to replicate certain input-output text generation patterns, without further fine-tuning. By feeding an LLM at inference time with a prompt that consists of a list of translation pairs, it can then simulate the domain and style characteristics. This work aims to investigate how we can utilize in-context learning to improve real-time adaptive MT. Our extensive experiments show promising results at translation time. For example, GPT-3.5 can adapt to a set of in-domain sentence pairs and/or terminology while translating a new sentence. We observe that the translation quality with few-shot in-context learning can surpass that of strong encoder-decoder MT systems, especially for high-resource languages. Moreover, we investigate whether we can combine MT from strong encoder-decoder models with fuzzy matches, which can further improve translation quality, especially for less supported languages. We conduct our experiments across five diverse language pairs, namely English-to-Arabic (EN-AR), English-to-Chinese (EN-ZH), English-to-French (EN-FR), English-to-Kinyarwanda (EN-RW), and English-to-Spanish (EN-ES).

Backward and Trigger-Based Language Models for Statistical Machine Translation

Enhancing Language Models in Statistical Machine Translation with Backward N-Grams and Mutual Information Triggers

Mutual Information and Diverse Decoding Improve Neural Machine Translation.

A Topic-Triggered Language Model for Statistical Machine Translation.

Improving Phrase-Based Statistical Machine Translation Models by Incorporating Syntax-Based Language Models

Incorporating Statistical Machine Translation Word Knowledge into Neural Machine Translation

An Approach to N-Gram Language Model Evaluation in Phrase-Based Statistical Machine Translation

A Method For Incorporating Language Models Based On Linkage Grammar Into Phrase-Based Smt Models

Modeling Past and Future for Neural Machine Translation

An Investigation On Statistical Machine Translation With Neural Language Models

Backward and Forward Language Modeling for Constrained Sentence Generation

A Sense-Based Translation Model For Statistical Machine Translation

Neural Network Based Bilingual Language Model Growing for Statistical Machine Translation

Neural Machine Translation Advised by Statistical Machine Translation

A Neural Approach to Source Dependence Based Context Model for Statistical Machine Translation

Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations

N-gram-based Tense Models for Statistical Machine Translation

Trigger-based language models: a maximum entropy approach

Improving Language Model Integration for Neural Machine Translation

Translation Model Combination Based On String-Chunk

Adaptive Machine Translation with Large Language Models