Abstract:With the accelerating development of Large Language Models (LLMs), many LLMs are beginning to be used in the Chinese K-12 education domain. The integration of LLMs and education is getting closer and closer, however, there is currently no benchmark for evaluating LLMs that focuses on the Chinese K-12 education domain. Therefore, there is an urgent need for a comprehensive natural language processing benchmark to accurately assess the capabilities of various LLMs in the Chinese K-12 education domain. To address this, we introduce the E-EVAL, the first comprehensive evaluation benchmark specifically designed for the Chinese K-12 education field. The E-EVAL consists of 4,351 multiple-choice questions at the primary, middle, and high school levels across a wide range of subjects, including Chinese, English, Politics, History, Ethics, Physics, Chemistry, Mathematics, and Geography. We conducted a comprehensive evaluation of E-EVAL on advanced LLMs, including both English-dominant and Chinese-dominant models. Findings show that Chinese-dominant models perform well compared to English-dominant models, with many scoring even above the GPT 4.0. However, almost all models perform poorly in complex subjects such as mathematics. We also found that most Chinese-dominant LLMs did not achieve higher scores at the primary school level compared to the middle school level. We observe that the mastery of higher-order knowledge by the model does not necessarily imply the mastery of lower-order knowledge as well. Additionally, the experimental results indicate that the Chain of Thought (CoT) technique is effective only for the challenging science subjects, while Few-shot prompting is more beneficial for liberal arts subjects. With E-EVAL, we aim to analyze the strengths and limitations of LLMs in educational applications, and to contribute to the progress and development of Chinese K-12 education and LLMs.

C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models

E-EVAL: A Comprehensive Chinese K-12 Education Evaluation Benchmark for Large Language Models

CJEval: A Benchmark for Assessing Large Language Models Using Chinese Junior High School Exam Data

FoundaBench: Evaluating Chinese Fundamental Knowledge Capabilities of Large Language Models

AC-EVAL: Evaluating Ancient Chinese Language Understanding in Large Language Models

F-Eval: Assessing Fundamental Abilities with Refined Evaluation Methods

OpenEval: Benchmarking Chinese LLMs across Capability, Alignment and Safety

F-Eval: Asssessing Fundamental Abilities with Refined Evaluation Methods

CLEVA: Chinese Language Models EVAluation Platform

SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Research

LHMKE: A Large-scale Holistic Multi-subject Knowledge Evaluation Benchmark for Chinese Large Language Models

TencentLLMEval: A Hierarchical Evaluation of Real-World Capabilities for Human-Aligned LLMs

An Improved Traditional Chinese Evaluation Suite for Foundation Model

CMMLU: Measuring massive multitask language understanding in Chinese

L-Eval: Instituting Standardized Evaluation for Long Context Language Models

CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Models

Benchmarking Large Language Models on CFLUE -- A Chinese Financial Language Understanding Evaluation Dataset

CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models

AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models

Evaluating the generation capabilities of large Chinese language models