Abstract:Large Language Models have gained considerable attention for their revolutionary capabilities. However, there is also growing concern on their safety implications, making a comprehensive safety evaluation for LLMs urgently needed before model deployment. In this work, we propose S-Eval, a new comprehensive, multi-dimensional and open-ended safety evaluation benchmark. At the core of S-Eval is a novel LLM-based automatic test prompt generation and selection framework, which trains an expert testing LLM Mt combined with a range of test selection strategies to automatically construct a high-quality test suite for the safety evaluation. The key to the automation of this process is a novel expert safety-critique LLM Mc able to quantify the riskiness score of an LLM's response, and additionally produce risk tags and explanations. Besides, the generation process is also guided by a carefully designed risk taxonomy with four different levels, covering comprehensive and multi-dimensional safety risks of concern. Based on these, we systematically construct a new and large-scale safety evaluation benchmark for LLMs consisting of 220,000 evaluation prompts, including 20,000 base risk prompts (10,000 in Chinese and 10,000 in English) and 200,000 corresponding attack prompts derived from 10 popular adversarial instruction attacks against LLMs. Moreover, considering the rapid evolution of LLMs and accompanied safety threats, S-Eval can be flexibly configured and adapted to include new risks, attacks and models. S-Eval is extensively evaluated on 20 popular and representative LLMs. The results confirm that S-Eval can better reflect and inform the safety risks of LLMs compared to existing benchmarks. We also explore the impacts of parameter scales, language environments, and decoding parameters on the evaluation, providing a systematic methodology for evaluating the safety of LLMs.

Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models

Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity

Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Models

CHiSafetyBench: A Chinese Hierarchical Safety Benchmark for Large Language Models

SafetyBench: Evaluating the Safety of Large Language Models with Multiple Choice Questions

SafetyBench: Evaluating the Safety of Large Language Models

SC-Safety: A Multi-round Open-ended Question Adversarial Safety Benchmark for Large Language Models in Chinese

ChineseSafe: A Chinese Benchmark for Evaluating Safety in Large Language Models

Safety Assessment of Chinese Large Language Models

CFSafety: Comprehensive Fine-grained Safety Assessment for LLMs

SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks

SAFETY-J: Evaluating Safety with Critique

LongSafetyBench: Long-Context LLMs Struggle with Safety Issues

All Languages Matter: On the Multilingual Safety of Large Language Models

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

S-Eval: Automatic and Adaptive Test Generation for Benchmarking Safety Evaluation of Large Language Models

A Chinese Dataset for Evaluating the Safeguards in Large Language Models

Long-form factuality in large language models

SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models

Measuring short-form factuality in large language models

LabSafety Bench: Benchmarking LLMs on Safety Issues in Scientific Labs