{"id":13754102,"url":"https://github.com/HowieHwong/TrustLLM","last_synced_at":"2025-05-09T22:30:49.388Z","repository":{"id":216263515,"uuid":"735121755","full_name":"HowieHwong/TrustLLM","owner":"HowieHwong","description":"[ICML 2024] TrustLLM: Trustworthiness in Large Language Models","archived":false,"fork":false,"pushed_at":"2024-09-29T02:57:52.000Z","size":11782,"stargazers_count":465,"open_issues_count":7,"forks_count":44,"subscribers_count":8,"default_branch":"main","last_synced_at":"2024-11-09T03:39:16.728Z","etag":null,"topics":["ai","benchmark","dataset","evaluation","large-language-models","llm","natural-language-processing","nlp","pypi-package","toolkit","trustworthy-ai","trustworthy-machine-learning"],"latest_commit_sha":null,"homepage":"https://trustllmbenchmark.github.io/TrustLLM-Website/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HowieHwong.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-23T18:32:57.000Z","updated_at":"2024-11-08T07:31:58.000Z","dependencies_parsed_at":"2024-06-19T03:01:12.912Z","dependency_job_id":"4f44e3c2-bc0f-412e-b954-7c7fe8796ba3","html_url":"https://github.com/HowieHwong/TrustLLM","commit_stats":null,"previous_names":["howiehwong/trustllm"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HowieHwong%2FTrustLLM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HowieHwong%2FTrustLLM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HowieHwong%2FTrustLLM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HowieHwong%2FTrustLLM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HowieHwong","download_url":"https://codeload.github.com/HowieHwong/TrustLLM/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224884612,"owners_count":17386121,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","benchmark","dataset","evaluation","large-language-models","llm","natural-language-processing","nlp","pypi-package","toolkit","trustworthy-ai","trustworthy-machine-learning"],"created_at":"2024-08-03T09:01:40.267Z","updated_at":"2025-05-09T22:30:49.381Z","avatar_url":"https://github.com/HowieHwong.png","language":"Python","funding_links":[],"categories":["A01_文本生成_文本对话","Evaluation and Monitoring","Python"],"sub_categories":["大语言对话模型及数据"],"readme":"\u003cdiv align=\"center\"\u003e\n\n\n\u003cimg src=\"https://raw.githubusercontent.com/TrustLLMBenchmark/TrustLLM-Website/main/img/logo.png\" width=\"100%\"\u003e\n\n# Toolkit for \"**TrustLLM: Trustworthiness in Large Language Models**\"\n\n\n[![Website](https://img.shields.io/badge/Website-%F0%9F%8C%8D-blue?style=for-the-badge\u0026logoWidth=40)](https://trustllmbenchmark.github.io/TrustLLM-Website/)\n[![Paper](https://img.shields.io/badge/Paper-%F0%9F%8E%93-lightgrey?style=for-the-badge\u0026logoWidth=40)](https://arxiv.org/abs/2401.05561)\n[![Dataset](https://img.shields.io/badge/Dataset-%F0%9F%92%BE-green?style=for-the-badge\u0026logoWidth=40)](https://huggingface.co/datasets/TrustLLM/TrustLLM-dataset)\n[![Data Map](https://img.shields.io/badge/Data%20Map-%F0%9F%8D%9F-orange?style=for-the-badge\u0026logoWidth=40)](https://atlas.nomic.ai/map/f64e87d3-c769-4a90-b15d-9dc833acc8ba/8e9d7045-503b-4ba0-bc64-7201cb7aacee?xs=-16.14086\u0026xf=-1.88776\u0026ys=-7.54937\u0026yf=3.88213)\n[![Leaderboard](https://img.shields.io/badge/Leaderboard-%F0%9F%9A%80-brightgreen?style=for-the-badge\u0026logoWidth=40)](https://trustllmbenchmark.github.io/TrustLLM-Website/leaderboard.html)\n[![Toolkit Document](https://img.shields.io/badge/Toolkit%20Document-%F0%9F%93%9A-blueviolet?style=for-the-badge\u0026logoWidth=40)](https://howiehwong.github.io/TrustLLM/)\n\n[![Downloads](https://static.pepy.tech/badge/trustllm)](https://pepy.tech/project/trustllm)\n[![Downloads](https://static.pepy.tech/badge/trustllm/month)](https://pepy.tech/project/trustllm)\n[![Downloads](https://static.pepy.tech/badge/trustllm/week)](https://pepy.tech/project/trustllm)\n\n\n\u003cimg src=\"https://img.shields.io/github/last-commit/HowieHwong/TrustLLM?style=flat-square\u0026color=5D6D7E\" alt=\"git-last-commit\" /\u003e\n\u003cimg src=\"https://img.shields.io/github/commit-activity/m/HowieHwong/TrustLLM?style=flat-square\u0026color=5D6D7E\" alt=\"GitHub commit activity\" /\u003e\n\u003cimg src=\"https://img.shields.io/github/languages/top/HowieHwong/TrustLLM?style=flat-square\u0026color=5D6D7E\" alt=\"GitHub top language\" /\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n\n\n\n\u003c/div\u003e\n\n\n## Updates \u0026 News\n\n- [02/20/2025] Our new work **TrustGen** and **TrustEval** toolkit has been released! [TrustGen](https://trustgen.github.io/) provides a comprehensive guidelines, assessment, and perspective for trustworthiness across multiple generative models, and [TrustEval](https://github.com/TrustGen/TrustEval-toolkit) offers a dynamic evaluation platform.\n\n- [01/09/2024] **TrustLLM** toolkit has been downloaded for 4000+ times!\n \n\u003cdetails\u003e\n\u003csummary\u003eClick to expand/collapse more\u003c/summary\u003e\n\n\n- [15/07/2024] **TrustLLM** now supports [**UniGen**](https://unigen-framework.github.io/) for dynamic evaluation.\n- [02/05/2024] 🥂 **TrustLLM has been accepted by ICML 2024! See you in Vienna!**\n- [23/04/2024] :star: Version 0.3.0: Major updates including bug fixes, enhanced evaluation, and new models added (including ChatGLM3, Llama3-8b, Llama3-70b, GLM4, Mixtral). ([See details](https://howiehwong.github.io/TrustLLM/changelog.html))\n- [20/03/2024] :star: Version 0.2.4: Fixed many bugs \u0026 Support Gemini Pro API\n- [01/02/2024] :page_facing_up: Version 0.2.2: See our new paper about the awareness in LLMs! ([link](https://arxiv.org/abs/2401.17882))\n- [29/01/2024] :star: Version 0.2.1: trustllm toolkit now supports (1) Easy evaluation pipeline (2) LLMs in [replicate](https://replicate.com/) and [deepinfra](https://deepinfra.com/) (3) [Azure OpenAI API](https://azure.microsoft.com/en-us/products/ai-services/openai-service)\n- [20/01/2024] :star: Version 0.2.0 of trustllm toolkit is released! See the [new features](https://howiehwong.github.io/TrustLLM/changelog.html#version-020).\n- [12/01/2024] :surfer: The [dataset](https://huggingface.co/datasets/TrustLLM/TrustLLM-dataset), [leaderboard](https://trustllmbenchmark.github.io/TrustLLM-Website/leaderboard.html), and [evaluation toolkit](https://howiehwong.github.io/TrustLLM/) are released!\n\n\u003c/details\u003e\n\n## 👂**TL;DR**\n\n- TrustLLM (ICML 2024) is a comprehensive framework for studying trustworthiness of large language models, which includes principles, surveys, and benchmarks.\n- This code repository is designed to provide an easy toolkit for evaluating the trustworthiness of LLMs ([See our docs](https://howiehwong.github.io/TrustLLM/)).\n\n\n\n**Table of Content**\n\n- [Toolkit for \"**TrustLLM: Trustworthiness in Large Language Models**\"](#toolkit-for-trustllm-trustworthiness-in-large-language-models)\n - [Updates \\\u0026 News](#updates--news)\n - [👂**TL;DR**](#tldr)\n - [🙋 **About TrustLLM**](#-about-trustllm)\n - [🧹 **Before Evaluation**](#-before-evaluation)\n - [**Installation**](#installation)\n - [**Dataset Download**](#dataset-download)\n - [**Generation**](#generation)\n - [🙌 **Evaluation**](#-evaluation)\n - [🛎️ **Dataset \\\u0026 Task**](#️-dataset--task)\n - [**Dataset overview:**](#dataset-overview)\n - [**Task overview:**](#task-overview)\n - [🏆 **Leaderboard**](#-leaderboard)\n - [📣 **Contribution**](#-contribution)\n - [**⏰ TODO in Coming Versions**](#-todo-in-coming-versions)\n - [**Citation**](#citation)\n - [**License**](#license)\n\n\n## 🙋 **About TrustLLM**\n\nWe introduce TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions. Specifically, we first propose a set of principles for trustworthy LLMs that span eight different dimensions. Based on these principles, we further establish a benchmark across six dimensions including truthfulness, safety, fairness, robustness, privacy, and machine ethics. \nWe then present a study evaluating 16 mainstream LLMs in TrustLLM, consisting of over 30 datasets. \nThe [document](https://howiehwong.github.io/TrustLLM/#about) explains how to use the trustllm python package to help you assess the performance of your LLM in trustworthiness more quickly. For more details about TrustLLM, please refer to [project website](https://trustllmbenchmark.github.io/TrustLLM-Website/).\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/TrustLLMBenchmark/TrustLLM-Website/main/img/benchmark_arch_00.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n\n\n\n## 🧹 **Before Evaluation**\n\n### **Installation**\nCreate a new environment:\n\n```shell\nconda create --name trustllm python=3.9\n```\n\n**Installation via Github (recommended):**\n\n```shell\ngit clone git@github.com:HowieHwong/TrustLLM.git\ncd TrustLLM/trustllm_pkg\npip install .\n```\n\n\n**Installation via `pip` (deprecated):**\n\n```shell\npip install trustllm\n```\n\n**Installation via `conda` (deprecated):**\n\n```sh\nconda install -c conda-forge trustllm\n```\n\n### **Dataset Download**\n\nDownload TrustLLM dataset:\n\n```python\nfrom trustllm.dataset_download import download_dataset\n\ndownload_dataset(save_path='save_path')\n```\n\n### **Generation**\n\nWe have added generation section from [version 0.2.0](https://howiehwong.github.io/TrustLLM/changelog.html). Start your generation from [this page](https://howiehwong.github.io/TrustLLM/guides/generation_details.html). Here is an example:\n\n```python\nfrom trustllm.generation.generation import LLMGeneration\n\nllm_gen = LLMGeneration(\n model_path=\"your model name\", \n test_type=\"test section\", \n data_path=\"your dataset file path\",\n model_name=\"\", \n online_model=False, \n use_deepinfra=False,\n use_replicate=False,\n repetition_penalty=1.0,\n num_gpus=1, \n max_new_tokens=512, \n debug=False,\n device='cuda:0'\n)\n\nllm_gen.generation_results()\n```\n\n\n## 🙌 **Evaluation**\n\nWe have provided a toolkit that allows you to more conveniently assess the trustworthiness of large language models. Please refer to [the document](https://howiehwong.github.io/TrustLLM/) for more details. Here is an example:\n\n```python\nfrom trustllm.task.pipeline import run_truthfulness\n\ntruthfulness_results = run_truthfulness( \n internal_path=\"path_to_internal_consistency_data.json\", \n external_path=\"path_to_external_consistency_data.json\", \n hallucination_path=\"path_to_hallucination_data.json\", \n sycophancy_path=\"path_to_sycophancy_data.json\",\n advfact_path=\"path_to_advfact_data.json\"\n)\n```\n\n## 🛎️ **Dataset \u0026 Task**\n\n### **Dataset overview:**\n\n*✓ the dataset is from prior work, and ✗ means the dataset is first proposed in our benchmark.*\n\n| Dataset | Description | Num. | Exist? | Section |\n|-----------------------|-----------------------------------------------------------------------------------------------------------------------|----------|--------|------------------------|\n| SQuAD2.0 | It combines questions in SQuAD1.1 with over 50,000 unanswerable questions. | 100 | ✓ | Misinformation |\n| CODAH | It contains 28,000 commonsense questions. | 100 | ✓ | Misinformation |\n| HotpotQA | It contains 113k Wikipedia-based question-answer pairs for complex multi-hop reasoning. | 100 | ✓ | Misinformation |\n| AdversarialQA | It contains 30,000 adversarial reading comprehension question-answer pairs. | 100 | ✓ | Misinformation |\n| Climate-FEVER | It contains 7,675 climate change-related claims manually curated by human fact-checkers. | 100 | ✓ | Misinformation |\n| SciFact | It contains 1,400 expert-written scientific claims pairs with evidence abstracts. | 100 | ✓ | Misinformation |\n| COVID-Fact | It contains 4,086 real-world COVID claims. | 100 | ✓ | Misinformation |\n| HealthVer | It contains 14,330 health-related claims against scientific articles. | 100 | ✓ | Misinformation |\n| TruthfulQA | The multiple-choice questions to evaluate whether a language model is truthful in generating answers to questions. | 352 | ✓ | Hallucination |\n| HaluEval | It contains 35,000 generated and human-annotated hallucinated samples. | 300 | ✓ | Hallucination |\n| LM-exp-sycophancy | A dataset consists of human questions with one sycophancy response example and one non-sycophancy response example. | 179 | ✓ | Sycophancy |\n| Opinion pairs | It contains 120 pairs of opposite opinions. | 240, 120 | ✗ | Sycophancy, Preference |\n| WinoBias | It contains 3,160 sentences, split for development and testing, created by researchers familiar with the project. | 734 | ✓ | Stereotype |\n| StereoSet | It contains the sentences that measure model preferences across gender, race, religion, and profession. | 734 | ✓ | Stereotype |\n| Adult | The dataset, containing attributes like sex, race, age, education, work hours, and work type, is utilized to predict salary levels for individuals. | 810 | ✓ | Disparagement |\n| Jailbreak Trigger | The dataset contains the prompts based on 13 jailbreak attacks. | 1300 | ✗ | Jailbreak, Toxicity |\n| Misuse (additional) | This dataset contains prompts crafted to assess how LLMs react when confronted by attackers or malicious users seeking to exploit the model for harmful purposes. | 261 | ✗ | Misuse |\n| Do-Not-Answer | It is curated and filtered to consist only of prompts to which responsible LLMs do not answer. | 344 + 95 | ✓ | Misuse, Stereotype |\n| AdvGLUE | A multi-task dataset with different adversarial attacks. | 912 | ✓ | Natural Noise |\n| AdvInstruction | 600 instructions generated by 11 perturbation methods. | 600 | ✗ | Natural Noise |\n| ToolE | A dataset with the users' queries which may trigger LLMs to use external tools. | 241 | ✓ | Out of Domain (OOD) |\n| Flipkart | A product review dataset, collected starting from December 2022. | 400 | ✓ | Out of Domain (OOD) |\n| DDXPlus | A 2022 medical diagnosis dataset comprising synthetic data representing about 1.3 million patient cases. | 100 | ✓ | Out of Domain (OOD) |\n| ETHICS | It contains numerous morally relevant scenarios descriptions and their moral correctness. | 500 | ✓ | Implicit Ethics |\n| Social Chemistry 101 | It contains various social norms, each consisting of an action and its label. | 500 | ✓ | Implicit Ethics |\n| MoralChoice | It consists of different contexts with morally correct and wrong actions. | 668 | ✓ | Explicit Ethics |\n| ConfAIde | It contains the description of how information is used. | 196 | ✓ | Privacy Awareness |\n| Privacy Awareness | It includes different privacy information queries about various scenarios. | 280 | ✗ | Privacy Awareness |\n| Enron Email | It contains approximately 500,000 emails generated by employees of the Enron Corporation. | 400 | ✓ | Privacy Leakage |\n| Xstest | It's a test suite for identifying exaggerated safety behaviors in LLMs. | 200 | ✓ | Exaggerated Safety |\n\n### **Task overview:**\n\n*○ means evaluation through the automatic scripts (e.g., keywords matching), ● means the automatic evaluation by ChatGPT, GPT-4 or longformer, and ◐ means the mixture evaluation.*\n\n*More trustworthy LLMs are expected to have a higher value of the metrics with ↑ and a lower value with ↓.*\n\n| Task Name | Metrics | Type | Eval | Section |\n|----------------------------------------------|-------------------------------------------|-----------------|------|--------------------------|\n| Closed-book QA | Accuracy (↑) | Generation | ○ | Misinformation(Internal) |\n| Fact-Checking | Macro F-1 (↑) | Classification | ● | Misinformation(External) |\n| Multiple Choice QA | Accuracy (↑) | Classification | ● | Hallucination |\n| Hallucination Classification | Accuracy (↑) | Classification | ● | Hallucination |\n| Persona Sycophancy | Embedding similarity (↑) | Generation | ◐ | Sycophancy |\n| Opinion Sycophancy | Percentage change (↓) | Generation | ○ | Sycophancy |\n| Factuality Correction | Percentage change (↑) | Generation | ○ | Adversarial Factuality |\n| Jailbreak Attack Evaluation | RtA (↑) | Generation | ○ | Jailbreak |\n| Toxicity Measurement | Toxicity Value (↓) | Generation | ● | Toxicity |\n| Misuse Evaluation | RtA (↑) | Generation | ○ | Misuse |\n| Exaggerated Safety Evaluation | RtA (↓) | Generation | ○ | Exaggerated Safety |\n| Agreement on Stereotypes | Accuracy (↑) | Generation | ◐ | Stereotype |\n| Recognition of Stereotypes | Agreement Percentage (↓) | Classification | ◐ | Stereotype |\n| Stereotype Query Test | RtA (↑) | Generation | ○ | Stereotype |\n| Preference Selection | RtA (↑) | Generation | ○ | Preference |\n| Salary Prediction | p-value (↑) | Generation | ● | Disparagement |\n| Adversarial Perturbation in Downstream Tasks | ASR (↓), RS (↑) | Generation | ◐ | Natural Noise |\n| Adversarial Perturbation in Open-Ended Tasks | Embedding similarity (↑) | Generation | ◐ | Natural Noise |\n| OOD Detection | RtA (↑) | Generation | ○ | Out of Domain (OOD) |\n| OOD Generalization | Micro F1 (↑) | Classification | ○ | Out of Domain (OOD) |\n| Agreement on Privacy Information | Pearson's correlation (↑) | Classification | ● | Privacy Awareness |\n| Privacy Scenario Test | RtA (↑) | Generation | ○ | Privacy Awareness |\n| Probing Privacy Information Usage | RtA (↑), Accuracy (↓) | Generation | ◐ | Privacy Leakage |\n| Moral Action Judgement | Accuracy (↑) | Classification | ◐ | Implicit Ethics |\n| Moral Reaction Selection (Low-Ambiguity) | Accuracy (↑) | Classification | ◐ | Explicit Ethics |\n| Moral Reaction Selection (High-Ambiguity) | RtA (↑) | Generation | ○ | Explicit Ethics |\n| Emotion Classification | Accuracy (↑) | Classification | ● | Emotional Awareness |\n\n## 🏆 **Leaderboard**\n\nIf you want to view the performance of all models or upload the performance of your LLM, please refer to [this link](https://trustllmbenchmark.github.io/TrustLLM-Website/leaderboard.html).\n\n![images/rank_card_00.png](images/rank_card_00.png \"ranking\")\n\n\n## 📣 **Contribution**\n\nWe welcome your contributions, including but not limited to the following:\n\n- New evaluation datasets\n- Research on trustworthy issues\n- Improvements to the toolkit\n\nIf you intend to make improvements to the toolkit, please fork the repository first, make the relevant modifications to the code, and finally initiate a `pull request`.\n\n## **⏰ TODO in Coming Versions**\n\n- [x] Faster and simpler evaluation pipeline (**Version 0.2.1**)\n- [x] Dynamic dataset ([UniGen](https://unigen-framework.github.io/))\n- [ ] More fine-grained datasets\n- [ ] Chinese output evaluation\n- [ ] Downstream application evaluation\n\n\n## **Citation**\n\n```text\n@inproceedings{huang2024trustllm,\n title={TrustLLM: Trustworthiness in Large Language Models},\n author={Yue Huang and Lichao Sun and Haoran Wang and Siyuan Wu and Qihui Zhang and Yuan Li and Chujie Gao and Yixin Huang and Wenhan Lyu and Yixuan Zhang and Xiner Li and Hanchi Sun and Zhengliang Liu and Yixin Liu and Yijue Wang and Zhikun Zhang and Bertie Vidgen and Bhavya Kailkhura and Caiming Xiong and Chaowei Xiao and Chunyuan Li and Eric P. Xing and Furong Huang and Hao Liu and Heng Ji and Hongyi Wang and Huan Zhang and Huaxiu Yao and Manolis Kellis and Marinka Zitnik and Meng Jiang and Mohit Bansal and James Zou and Jian Pei and Jian Liu and Jianfeng Gao and Jiawei Han and Jieyu Zhao and Jiliang Tang and Jindong Wang and Joaquin Vanschoren and John Mitchell and Kai Shu and Kaidi Xu and Kai-Wei Chang and Lifang He and Lifu Huang and Michael Backes and Neil Zhenqiang Gong and Philip S. Yu and Pin-Yu Chen and Quanquan Gu and Ran Xu and Rex Ying and Shuiwang Ji and Suman Jana and Tianlong Chen and Tianming Liu and Tianyi Zhou and William Yang Wang and Xiang Li and Xiangliang Zhang and Xiao Wang and Xing Xie and Xun Chen and Xuyu Wang and Yan Liu and Yanfang Ye and Yinzhi Cao and Yong Chen and Yue Zhao},\n booktitle={Forty-first International Conference on Machine Learning},\n year={2024},\n url={https://openreview.net/forum?id=bWUU0LwwMp}\n}\n```\n\n\n[//]: # (## Star History)\n\n[//]: # ()\n[//]: # ([![Star History Chart]\u0026#40;https://api.star-history.com/svg?repos=HowieHwong/TrustLLM\u0026type=Date\u0026#41;]\u0026#40;https://star-history.com/#HowieHwong/TrustLLM\u0026Date\u0026#41;)\n\n\n\n## **License**\n\nThe code in this repository is open source under the [MIT license](https://github.com/HowieHwong/TrustLLM/blob/main/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHowieHwong%2FTrustLLM","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHowieHwong%2FTrustLLM","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHowieHwong%2FTrustLLM/lists"}