Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/thu-coai/SafetyBench

Official github repo for SafetyBench, a comprehensive benchmark to evaluate LLMs' safety.
https://github.com/thu-coai/SafetyBench

Last synced: about 1 month ago
JSON representation

Official github repo for SafetyBench, a comprehensive benchmark to evaluate LLMs' safety.

Awesome Lists containing this project

README 

        


SafetyBench






   🌐 Website • 🤗 Hugging Face • ⏬ Data •   📃 Paper




SafetyBench is a comprehensive benchmark for evaluating the safety of LLMs, which comprises 11,435 diverse multiple choice questions spanning across 7 distinct categories of safety concerns. SafetyBench also incorporates both Chinese and English data, facilitating the evaluation in both languages. Please visit our [website](https://llmbench.ai/safety) or check our [paper](https://arxiv.org/abs/2309.07045) for more details. 



![SafetyBench](./figs/overview.png)



## News

**🎉 `2024/06/24`:** SafetyBench has been accepted by the ACL 2024 Main Conference. See our camera-ready version of the paper at [Arxiv](https://arxiv.org/abs/2309.07045). SafetyBench is also integrated into [SuperBench](https://fm.ai.tsinghua.edu.cn/superbench/#/).



## Table of Contents 

- [Leaderboard](#leaderboard)

- [Data](#data)

  - [Download](#download)

  - [Description](#description)

- [How to Evaluate on SafetyBench](#how-to-evaluate-on-safetybench)

- [How to Submit](#how-to-submit)

- [Citation](#citation)



## Leaderboard

The up-to-date leaderboards are on our [website](https://llmbench.ai/safety). We have three leaderboards for Chinese, English and Chinese subset respectively. We remove questions with highly sensitive keywords and downsample 300 questions for each category to construct the Chinese subset. Summarized evaluation results of some representative LLMs are shown below:



![Result](./figs/res.png)



## Data

### Download

We put our data on the [Hugging Face website](https://huggingface.co/datasets/thu-coai/SafetyBench).



You can download the test questions and few-shot examples through `wget` directly. Just run the script [`download_data.sh`](./code/download_data.sh)



Alternatively, you can download the test questions and few-shot examples through the `datasets` library. Just run the code [`download_data.py`](./code/download_data.py)



### Description

`test_zh`, `test_en` and `test_zh_subset` contain test questions for Chinese, English and Chinese subset respectively. `dev_zh` and `dev_en` contain 5 examples for each safety category, which can be used as few-shot demonstrations. 



Note that the `options` field in the data includes at most four items, corresponding to the options A, B, C, D in order. For the `answer` field in the dev data, the mapping rule is: 0->A, 1->B, 2->C, 3->D.



## How to Evaluate on SafetyBench

In our paper, we conduct experiments in both zero-shot and five-shot settings. And we extract the predicted answers from models' responses. An example of evaluation code could be found at [code](./code/evaluate_baichuan.py). We don’t include CoT-based evaluation because SafetyBench is less reasoning-intensive than benchmarks testing the model’s

general capabilities such as MMLU. But feel free to submit your results based on CoT. The default prompt for zero-shot and five-shot evaluation is shown below:

![figure](./figs/eva_prompts.png)



To enable more accurate extraction of the predicted answers, we made minor changes to the prompts for some models, which is shown below:

![figure](./figs/prompt_change.png)



## How to Submit

You need to first prepare a UTF-8 encoded JSON file with the following format, please refer to [submission_example.json](./submission_example.json) for details.



```

## key is the "id" field of the test questions

## value is the predicted answer: 0->A, 1->B, 2->C, 3->D

{

    "0": 0,

    "1": 1,

    "2": 3,

    "3": 2 

}

```



Then you can submit the JSON file to our [website](https://llmbench.ai/safety). 



## Citation

```

@article{zhang2023safetybench,

      title={SafetyBench: Evaluating the Safety of Large Language Models with Multiple Choice Questions}, 

      author={Zhexin Zhang and Leqi Lei and Lindong Wu and Rui Sun and Yongkang Huang and Chong Long and Xiao Liu and Xuanyu Lei and Jie Tang and Minlie Huang},

      journal={arXiv preprint arXiv:2309.07045},

      year={2023}

}

```