SafetyBench: Evaluating the Safety of Large Language Models

Zhexin Zhang,Leqi Lei,Lindong Wu,Rui Sun,Yongkang Huang,Chong Long,Xiao Liu,Xuanyu Lei,Jie Tang,Minlie Huang

2024-06-24

Abstract:With the rapid development of Large Language Models (LLMs), increasing attention has been paid to their safety concerns. Consequently, evaluating the safety of LLMs has become an essential task for facilitating the broad applications of LLMs. Nevertheless, the absence of comprehensive safety evaluation benchmarks poses a significant impediment to effectively assess and enhance the safety of LLMs. In this work, we present SafetyBench, a comprehensive benchmark for evaluating the safety of LLMs, which comprises 11,435 diverse multiple choice questions spanning across 7 distinct categories of safety concerns. Notably, SafetyBench also incorporates both Chinese and English data, facilitating the evaluation in both languages. Our extensive tests over 25 popular Chinese and English LLMs in both zero-shot and few-shot settings reveal a substantial performance advantage for GPT-4 over its counterparts, and there is still significant room for improving the safety of current LLMs. We also demonstrate that the measured safety understanding abilities in SafetyBench are correlated with safety generation abilities. Data and evaluation guidelines are available at \url{<a class="link-external link-https" href="https://github.com/thu-coai/SafetyBench" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://github.com/thu-coai/SafetyBench" rel="external noopener nofollow">this https URL</a>}. Submission entrance and leaderboard are available at \url{<a class="link-external link-https" href="https://llmbench.ai/safety" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://llmbench.ai/safety" rel="external noopener nofollow">this https URL</a>}.

Computation and Language

What problem does this paper attempt to address?

The paper aims to address the issue of safety evaluation for large language models (LLMs). Specifically, the researchers developed a benchmark tool called SafetyBench to comprehensively assess the performance of LLMs across different categories of safety issues. SafetyBench includes 11,435 multiple-choice questions, covering seven different categories of safety issues, and supports bilingual data in both Chinese and English to better evaluate the cross-linguistic safety of LLMs. By comparing various popular LLMs, it was found that GPT-4 significantly outperforms other models in zero-shot and few-shot settings. However, it also points out that there is still considerable room for improvement in the safety of current LLMs. Additionally, the paper verifies the correlation between the safety understanding ability and the safety generation ability measured by SafetyBench.

SafetyBench: Evaluating the Safety of Large Language Models

SafetyBench: Evaluating the Safety of Large Language Models with Multiple Choice Questions

Safety Assessment of Chinese Large Language Models

CHiSafetyBench: A Chinese Hierarchical Safety Benchmark for Large Language Models

LabSafety Bench: Benchmarking LLMs on Safety Issues in Scientific Labs

CFSafety: Comprehensive Fine-grained Safety Assessment for LLMs

SC-Safety: A Multi-round Open-ended Question Adversarial Safety Benchmark for Large Language Models in Chinese

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models

LongSafetyBench: Long-Context LLMs Struggle with Safety Issues

SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models

Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models

SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks

All Languages Matter: On the Multilingual Safety of Large Language Models

ChineseSafe: A Chinese Benchmark for Evaluating Safety in Large Language Models

S-Eval: Automatic and Adaptive Test Generation for Benchmarking Safety Evaluation of Large Language Models

LawBench: Benchmarking Legal Knowledge of Large Language Models

SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types

SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Behaviors

A Chinese Dataset for Evaluating the Safeguards in Large Language Models

MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Models