Abstract:Consistency is a key requirement of high-quality translation. It is especially important to adhere to pre-approved terminology and adapt to corrected translations in domain-specific projects. Machine translation (MT) has achieved significant progress in the area of domain adaptation. However, real-time adaptation remains challenging. Large-scale language models (LLMs) have recently shown interesting capabilities of in-context learning, where they learn to replicate certain input-output text generation patterns, without further fine-tuning. By feeding an LLM at inference time with a prompt that consists of a list of translation pairs, it can then simulate the domain and style characteristics. This work aims to investigate how we can utilize in-context learning to improve real-time adaptive MT. Our extensive experiments show promising results at translation time. For example, GPT-3.5 can adapt to a set of in-domain sentence pairs and/or terminology while translating a new sentence. We observe that the translation quality with few-shot in-context learning can surpass that of strong encoder-decoder MT systems, especially for high-resource languages. Moreover, we investigate whether we can combine MT from strong encoder-decoder models with fuzzy matches, which can further improve translation quality, especially for less supported languages. We conduct our experiments across five diverse language pairs, namely English-to-Arabic (EN-AR), English-to-Chinese (EN-ZH), English-to-French (EN-FR), English-to-Kinyarwanda (EN-RW), and English-to-Spanish (EN-ES).

Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution

Entropy-Based Vocabulary Substitution for Incremental Learning in Multilingual Neural Machine Translation

Continuous Learning in Neural Machine Translation using Bilingual Dictionaries

Adapters for Altering LLM Vocabularies: What Languages Benefit the Most?

Continual Learning for Neural Machine Translation

Vocabulary Manipulation for Neural Machine Translation

Extending Multilingual Machine Translation through Imitation Learning

Adaptive Machine Translation with Large Language Models

Learn and Consolidate: Continual Adaptation for Zero-Shot and Multilingual Neural Machine Translation.

Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages

Incorporating Bilingual Dictionaries for Low Resource Semi-Supervised Neural Machine Translation

Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation

Vocabulary Adaptation for Distant Domain Adaptation in Neural Machine Translation

Continual Learning Under Language Shift

Augmenting Statistical Machine Translation with Subword Translation of Out-of-Vocabulary Words

Vocabulary Learning Via Optimal Transport for Neural Machine Translation

Bilingual Adaptation of Monolingual Foundation Models

Continual Learning for Multilingual Neural Machine Translation Via Dual Importance-based Model Division

Improving Multimodal Large Language Models Using Continual Learning

Cross-lingual Lifelong Learning

On Using Very Large Target Vocabulary for Neural Machine Translation