EXAONE 3.5: Series of Large Language Models for Real-world Use Cases

LG AI Research,Soyoung An,Kyunghoon Bae,Eunbi Choi,Kibong Choi,Stanley Jungkyu Choi,Seokhee Hong,Junwon Hwang,Hyojin Jeon,Gerrard Jeongwon Jo,Hyunjik Jo,Jiyeon Jung,Yountae Jung,Hyosang Kim,Joonkee Kim,Seonghwan Kim,Soyeon Kim,Sunkyoung Kim,Yireun Kim,Yongil Kim,Youchul Kim,Edward Hwayoung Lee,Haeju Lee,Honglak Lee,Jinsik Lee,Kyungmin Lee,Woohyung Lim,Sangha Park,Sooyoun Park,Yongmin Park,Sihoon Yang,Heuiyeen Yeen,Hyeongu Yun

2024-12-06

Abstract:This technical report introduces the EXAONE 3.5 instruction-tuned language models, developed and released by LG AI Research. The EXAONE 3.5 language models are offered in three configurations: 32B, 7.8B, and 2.4B. These models feature several standout capabilities: 1) exceptional instruction following capabilities in real-world scenarios, achieving the highest scores across seven benchmarks, 2) outstanding long-context comprehension, attaining the top performance in four benchmarks, and 3) competitive results compared to state-of-the-art open models of similar sizes across nine general benchmarks. The EXAONE 3.5 language models are open to anyone for research purposes and can be downloaded from <a class="link-external link-https" href="https://huggingface.co/LGAI-EXAONE" rel="external noopener nofollow">this https URL</a>. For commercial use, please reach out to the official contact point of LG AI Research: contact_us@lgresearch.ai.

Computation and Language

What problem does this paper attempt to address?

This paper attempts to solve the following problems:

1. **Meeting Diverse User Needs**:
- The academic community has given feedback that small - scale models that can be trained and deployed on low - spec GPUs are required, because many researchers do not have access to advanced computing resources.
- The industrial sector hopes for larger models with stronger performance but still cost - effective, as well as small - scale models suitable for device - side deployment.
- With the wide application of Retrieval - Augmented Generation (RAG) technology, users' demand for models that can effectively handle long - context has increased.

2. **Improving Model Performance**:
- Improve the model's ability to follow instructions in real - world scenarios, enabling it to better understand and execute diverse user instructions.
- Enhance the model's understanding ability of long - context and ensure its better performance in handling complex tasks.
- Remain competitive with existing state - of - the - art models in general fields, especially in aspects such as mathematics, programming, and knowledge embedding.

3. **Data Compliance and Ethical Issues**:
- Developing AI models requires a large amount of data, and the acquisition and use of data may lead to issues such as copyright, intellectual property rights, and personal information protection. Therefore, LG AI Research has carried out AI compliance reviews throughout the data collection, model training, and information provision processes to minimize these risks.

4. **Model Release and Application**:
- To support researchers in promoting research on generative AI and developing innovative applications, the EXAONE 3.5 language model series is open for all researchers to download for non - commercial purposes. At the same time, it is also hoped that through the release of these models, the development of AI technology can be promoted and human life can be improved.

In summary, this paper aims to meet the diverse needs of the academic and industrial sectors by introducing a series of language models of different scales (from 2.4 billion to 32 billion parameters), improve the performance of models in various application scenarios, and ensure data compliance and responsible AI development.