Abstract:Large language models (LLMs), with their powerful generative capabilities and vast knowledge, empower various tasks in everyday life. However, these abilities are primarily concentrated in high-resource languages, leaving low-resource languages with weaker generative capabilities and relatively limited knowledge. Enhancing the multilingual capabilities of LLMs is therefore crucial for serving over 100 linguistic communities worldwide. An intuitive approach to enhance the multilingual capabilities would be to construct instruction data for various languages, but constructing instruction data for over 100 languages is prohibitively costly. In this paper, we introduce BayLing 2, which efficiently transfers generative capabilities and knowledge from high-resource languages to low-resource languages through language alignment. To achieve this, we constructed a dataset of 3.2 million instructions, comprising high-resource language instructions (Chinese and English) and cross-lingual instructions for 100+ languages and performed instruction tuning based on the dataset to facilitate the capability transfer between languages. Using Llama as the foundation model, we developed BayLing-2-7B, BayLing-2-13B, and BayLing-3-8B, and conducted a comprehensive evaluation of BayLing. For multilingual translation across 100+ languages, BayLing shows superior performance compared to open-source models of similar scale. For multilingual knowledge and understanding benchmarks, BayLing achieves significant improvements across over 20 low-resource languages, demonstrating its capability of effective knowledge transfer from high-resource to low-resource languages. Furthermore, results on English benchmarks indicate that BayLing maintains high performance in highresource languages while enhancing the performance in low-resource languages. Demo, homepage, code and models of BayLing are available.

Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners

Extrapolating Large Language Models to Non-English by Aligning Languages

The Reasonableness Behind Unreasonable Translation Capability of Large Language Model

Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models

How do Large Language Models Handle Multilingualism?

A Novel Paradigm Boosting Translation Capabilities of Large Language Models

Language Imbalance Driven Rewarding for Multilingual Self-improving

BayLing 2: A Multilingual Large Language Model with Efficient Language Alignment

Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis

Lens: Rethinking Multilingual Enhancement for Large Language Models

Improving Bilingual Capabilities of Language Models to Support Diverse Linguistic Practices in Education

Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models

How Vocabulary Sharing Facilitates Multilingualism in LLaMA?

Do Large Language Models Have an English Accent? Evaluating and Improving the Naturalness of Multilingual LLMs

A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias

Could We Have Had Better Multilingual LLMs If English Was Not the Central Language?

LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback

Towards Boosting Many-to-Many Multilingual Machine Translation with Large Language Models

Aligning Large Language Models for Controllable Recommendations

RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs