Abstract:Large Language Models have gained considerable attention for their revolutionary capabilities. However, there is also growing concern on their safety implications, making a comprehensive safety evaluation for LLMs urgently needed before model deployment. In this work, we propose S-Eval, a new comprehensive, multi-dimensional and open-ended safety evaluation benchmark. At the core of S-Eval is a novel LLM-based automatic test prompt generation and selection framework, which trains an expert testing LLM Mt combined with a range of test selection strategies to automatically construct a high-quality test suite for the safety evaluation. The key to the automation of this process is a novel expert safety-critique LLM Mc able to quantify the riskiness score of an LLM's response, and additionally produce risk tags and explanations. Besides, the generation process is also guided by a carefully designed risk taxonomy with four different levels, covering comprehensive and multi-dimensional safety risks of concern. Based on these, we systematically construct a new and large-scale safety evaluation benchmark for LLMs consisting of 220,000 evaluation prompts, including 20,000 base risk prompts (10,000 in Chinese and 10,000 in English) and 200,000 corresponding attack prompts derived from 10 popular adversarial instruction attacks against LLMs. Moreover, considering the rapid evolution of LLMs and accompanied safety threats, S-Eval can be flexibly configured and adapted to include new risks, attacks and models. S-Eval is extensively evaluated on 20 popular and representative LLMs. The results confirm that S-Eval can better reflect and inform the safety risks of LLMs compared to existing benchmarks. We also explore the impacts of parameter scales, language environments, and decoding parameters on the evaluation, providing a systematic methodology for evaluating the safety of LLMs.

SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models

SafetyBench: Evaluating the Safety of Large Language Models

SafetyBench: Evaluating the Safety of Large Language Models with Multiple Choice Questions

CHiSafetyBench: A Chinese Hierarchical Safety Benchmark for Large Language Models

SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks

S-Eval: Automatic and Adaptive Test Generation for Benchmarking Safety Evaluation of Large Language Models

ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming

Agent-SafetyBench: Evaluating the Safety of LLM Agents

LongSafetyBench: Long-Context LLMs Struggle with Safety Issues

SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Behaviors

LabSafety Bench: Benchmarking LLMs on Safety Issues in Scientific Labs

Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward

SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models

MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Models

SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types

CFSafety: Comprehensive Fine-grained Safety Assessment for LLMs

ChemSafetyBench: Benchmarking LLM Safety on Chemistry Domain

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Models

ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors

MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models