Abstract:Large Language Models have gained considerable attention for their revolutionary capabilities. However, there is also growing concern on their safety implications, making a comprehensive safety evaluation for LLMs urgently needed before model deployment. In this work, we propose S-Eval, a new comprehensive, multi-dimensional and open-ended safety evaluation benchmark. At the core of S-Eval is a novel LLM-based automatic test prompt generation and selection framework, which trains an expert testing LLM Mt combined with a range of test selection strategies to automatically construct a high-quality test suite for the safety evaluation. The key to the automation of this process is a novel expert safety-critique LLM Mc able to quantify the riskiness score of an LLM's response, and additionally produce risk tags and explanations. Besides, the generation process is also guided by a carefully designed risk taxonomy with four different levels, covering comprehensive and multi-dimensional safety risks of concern. Based on these, we systematically construct a new and large-scale safety evaluation benchmark for LLMs consisting of 220,000 evaluation prompts, including 20,000 base risk prompts (10,000 in Chinese and 10,000 in English) and 200,000 corresponding attack prompts derived from 10 popular adversarial instruction attacks against LLMs. Moreover, considering the rapid evolution of LLMs and accompanied safety threats, S-Eval can be flexibly configured and adapted to include new risks, attacks and models. S-Eval is extensively evaluated on 20 popular and representative LLMs. The results confirm that S-Eval can better reflect and inform the safety risks of LLMs compared to existing benchmarks. We also explore the impacts of parameter scales, language environments, and decoding parameters on the evaluation, providing a systematic methodology for evaluating the safety of LLMs.

WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Models

S-Eval: Automatic and Adaptive Test Generation for Benchmarking Safety Evaluation of Large Language Models

ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming

CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models

SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

Safety Assessment of Chinese Large Language Models

SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models

SafetyBench: Evaluating the Safety of Large Language Models

SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models

SafetyBench: Evaluating the Safety of Large Language Models with Multiple Choice Questions

Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

CFSafety: Comprehensive Fine-grained Safety Assessment for LLMs

OpenEval: Benchmarking Chinese LLMs across Capability, Alignment and Safety

All Languages Matter: On the Multilingual Safety of Large Language Models

A Chinese Dataset for Evaluating the Safeguards in Large Language Models

CS-Eval: A Comprehensive Large Language Model Benchmark for CyberSecurity

Evaluating Large Language Models: A Comprehensive Survey

Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward

ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors

SAFETY-J: Evaluating Safety with Critique