Abstract:The advent of Multilingual Language Models (MLLMs) and Large Language Models has spawned innovation in many areas of natural language processing. Despite the exciting potential of this technology, its impact on developing high-quality Machine Translation (MT) outputs for low-resource languages remains relatively under-explored. Furthermore, an open-source application, dedicated to both fine-tuning MLLMs and managing the complete MT workflow for low-resources languages, remains unavailable. We aim to address these imbalances through the development of adaptMLLM, which streamlines all processes involved in the fine-tuning of MLLMs for MT. This open-source application is tailored for developers, translators, and users who are engaged in MT. An intuitive interface allows for easy customisation of hyperparameters, and the application offers a range of metrics for model evaluation and the capability to deploy models as a translation service directly within the application. As a multilingual tool, we used adaptMLLM to fine-tune models for two low-resource language pairs: English to Irish (EN$\leftrightarrow$GA) and English to Marathi (EN$\leftrightarrow$MR). Compared with baselines from the LoResMT2021 Shared Task, the adaptMLLM system demonstrated significant improvements. In the EN$\rightarrow$GA direction, an improvement of 5.2 BLEU points was observed and an increase of 40.5 BLEU points was recorded in the GA$\rightarrow$EN direction. Significant improvements in the translation performance of the EN$\leftrightarrow$MR pair were also observed notably in the MR$\rightarrow$EN direction with an increase of 21.3 BLEU points. Finally, a fine-grained human evaluation of the MLLM output on the EN$\rightarrow$GA pair was conducted using the Multidimensional Quality Metrics and Scalar Quality Metrics error taxonomies. The application and models are freely available.

What problem does this paper attempt to address?

This paper focuses on how to utilize an integrated large language model (LLM) playground to fine-tune multilingual language models (MLLM) for low-resource language machine translation (MT). Currently, although multilingual and large language models have brought innovation to the field of natural language processing, their potential in improving high-quality machine translation output for low-resource languages has not been fully explored. In addition, there is a lack of an open-source application specifically for fine-tuning MLLM and managing the complete MT workflow for these low-resource languages. The paper introduces an open-source application called adaptMLLM, which simplifies the fine-tuning process of MLLM and is particularly user-friendly for developers, translators, and users, especially for beginners in this field. It provides an intuitive interface for easy customization of hyperparameters and includes various metrics for model evaluation, as well as the ability to deploy models as translation services directly within the application. AdaptMLLM adapts MLLM for two pairs of low-resource languages (English to Irish and English to Marathi) and demonstrates significant improvements in the baseline of the LoResMT2021 shared task. Furthermore, human evaluation is conducted, using both multi-dimensional quality metrics and scalar quality metric error classification, to further analyze the quality of model outputs. The paper is well-organized, starting with an introduction to related work and background information, then describing the dataset used, followed by a detailed discussion of key features of the adaptMLLM architecture. Empirical evaluations, including human evaluation, are conducted, and the system is discussed. Finally, the paper concludes and proposes future directions for work. The main contribution of the paper is providing a convenient and effective tool to enhance machine translation performance for low-resource languages.

adaptMLLM: Fine-Tuning Multilingual Language Models on Low-Resource Languages with Integrated LLM Playgrounds

Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustments

Meta-Learning for Low-Resource Neural Machine Translation.

Adaptive Machine Translation with Large Language Models

IntGrad MT: Eliciting LLMs' Machine Translation Capabilities with Sentence Interpolation and Gradual MT

Fine-tuning Large Language Models for Adaptive Machine Translation

Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translation

Leveraging LLMs for MT in Crisis Scenarios: a blueprint for low-resource languages

Beyond Metrics: Evaluating LLMs' Effectiveness in Culturally Nuanced, Low-Resource Real-World Scenarios

A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models

Adapting Large Language Models for Document-Level Machine Translation

Simul-LLM: A Framework for Exploring High-Quality Simultaneous Translation with Large Language Models

Challenges in Adapting Multilingual LLMs to Low-Resource Languages using LoRA PEFT Tuning

A Three-Pronged Approach to Cross-Lingual Adaptation with Multilingual LLMs

X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale

Targeted Multilingual Adaptation for Low-resource Language Families

MM-Eval: A Multilingual Meta-Evaluation Benchmark for LLM-as-a-Judge and Reward Models

Fine-tuning Large Language Models for Domain-specific Machine Translation

Assessing Translation capabilities of Large Language Models involving English and Indian Languages

From LLM to NMT: Advancing Low-Resource Machine Translation with Claude

Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis