https://github.com/osu-nlp-group/eval-llm-trust
https://github.com/osu-nlp-group/eval-llm-trust
Last synced: over 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/osu-nlp-group/eval-llm-trust
- Owner: OSU-NLP-Group
- Created: 2023-11-15T20:07:17.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-04-07T16:32:14.000Z (over 2 years ago)
- Last Synced: 2025-04-04T10:03:36.549Z (over 1 year ago)
- Language: Python
- Size: 19.4 MB
- Stars: 6
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities
This is the repo for our [work](https://arxiv.org/abs/2311.09447) to assess the trustworthiness of open-source LLMs.
The rapid progress in open-source Large Language Models (LLMs) is significantly driving AI development forward. However, there is still a limited understanding of their trustworthiness. Deploying these models at scale without sufficient trustworthiness can pose significant risks, highlighting the need to uncover these issues promptly. In this work, we conduct an adversarial assessment of open-source LLMs on trustworthiness, scrutinizing them across eight different aspects including toxicity, stereotypes, ethics, hallucination, fairness, sycophancy, privacy, and robustness against adversarial demonstrations. We propose **advCoU**, an extended Chain of Utterances-based (CoU) prompting strategy by incorporating carefully crafted malicious demonstrations for trustworthiness attack. Our extensive experiments encompass recent and representative series of open-source LLMs, including Vicuna, MPT, Falcon, Mistral, and Llama 2. The empirical outcomes underscore the efficacy of our attack strategy across diverse aspects. More interestingly, *our result analysis reveals that models with superior performance in general NLP tasks do not always have greater trustworthiness; in fact, larger models can be more vulnerable to attacks. Additionally, models that have undergone instruction tuning, focusing on instruction following, tend to be more susceptible, although fine-tuning LLMs for safety alignment proves effective in mitigating adversarial trustworthiness attacks.*
## Files
`generation.py:` The main function to run attack.
`load_data.py:` Load the dataset for each aspect for attack.
`prompts.py:` Prompt template for each aspect.
`evaluate.py:` The main function to run evaluation.
`./datasets:` This folder contains the datasets for all aspects.
`./output:` This folder stores the results generated by LLMs.
## Run
The command to run the generation:
```bash
python generate.py \
--method ours \
--model meta-llama/Llama-2-7b-chat-hf \
--aspect toxicity \
--turn single
```
Key Descriptions:
- `method`: Specify the attack strategy. "ours" denotes using our own attack strategy.
- `model`: The name of the model to attack, such as "meta-llama/Llama-2-7b-chat-hf", "lmsys/vicuna-7b-v1.3", etc.
- `aspect`: Specify the aspect to attack, including "toxicity", "stereotype", "ethics", "hallucination", "fairness", "sycophancy", "privacy", "robustness".
- `turn`: Specify the attack setting: "single" or "multi". Only use "single" for now.
## Evaluate
The command to run the generation:
```bash
python evaluate.py \
--method ours \
--model meta-llama/Llama-2-7b-chat-hf \
--aspect toxicity \
--turn single
```
Key Descriptions:
- `method`: Specify the attack strategy. "ours" denotes using our own attack strategy.
- `model`: The name of the model to attack, such as "meta-llama/Llama-2-7b-chat-hf", "lmsys/vicuna-7b-v1.3", etc.
- `aspect`: Specify the aspect to attack, including "toxicity", "stereotype", "ethics", "hallucination", "fairness", "sycophancy", "privacy", "robustness".
- `turn`: Specify the attack setting: "single" or "multi". Only use "single" for now.
## Citation
```
@misc{mo2023trustworthy,
title={How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities},
author={Lingbo Mo and Boshi Wang and Muhao Chen and Huan Sun},
year={2023},
eprint={2311.09447},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```