Abstract:The large language model (LLM) has garnered significant attention due to its in-context learning mechanisms and emergent capabilities. The research community has conducted several pilot studies to apply LLMs to machine translation tasks and evaluate their performance from diverse perspectives. However, previous research has primarily focused on the LLM itself and has not explored human intervention in the inference process of LLM. The characteristics of LLM, such as in-context learning and prompt engineering, closely mirror human cognitive abilities in language tasks, offering an intuitive solution for human-in-the-loop generation. In this study, we propose a human-in-the-loop pipeline that guides LLMs to produce customized outputs with revision instructions. The pipeline initiates by prompting the LLM to produce a draft translation, followed by the utilization of automatic retrieval or human feedback as supervision signals to enhance the LLM's translation through in-context learning. The human-machine interactions generated in this pipeline are also stored in an external database to expand the in-context retrieval database, enabling us to leverage human supervision in an offline setting. We evaluate the proposed pipeline using GPT-3.5-turbo API on five domain-specific benchmarks for German-English translation. The results demonstrate the effectiveness of the pipeline in tailoring in-domain translations and improving translation performance compared to direct translation. Additionally, we discuss the results from the following perspectives: 1) the effectiveness of different in-context retrieval methods; 2) the construction of a retrieval database under low-resource scenarios; 3) the observed domains differences; 4) the quantitative analysis of linguistic statistics; and 5) the qualitative analysis of translation cases. The code and data are available at <a class="link-external link-https" href="https://github.com/NLP2CT/HIL-MT/" rel="external noopener nofollow">this https URL</a>.

Hire a Linguist!: Learning Endangered Languages with In-Context Linguistic Descriptions

Can we teach language models to gloss endangered languages?

Teaching Large Language Models an Unseen Language on the Fly

Can Large Language Models Code Like a Linguist?: A Case Study in Low Resource Sound Law Induction

Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts

Supervised Knowledge Makes Large Language Models Better In-context Learners

Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?

Learning-From-Mistakes Prompting for Indigenous Language Translation

Human-in-the-loop Machine Translation with Large Language Model

BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Models

SambaLingo: Teaching Large Language Models New Languages

Bridging the Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs

LLM for Everyone: Representing the Underrepresented in Large Language Models

Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners

A Three-Pronged Approach to Cross-Lingual Adaptation with Multilingual LLMs

Towards Effective Disambiguation for Machine Translation with Large Language Models

Understanding and Mitigating Language Confusion in LLMs

BayLing 2: A Multilingual Large Language Model with Efficient Language Alignment

Walking a Tightrope -- Evaluating Large Language Models in High-Risk Domains

An Application of Large Language Models to Coding Negotiation Transcripts

Improving LLM Abilities in Idiomatic Translation