{"id":19383942,"url":"https://github.com/pku-yuangroup/hallucination-attack","last_synced_at":"2025-04-23T21:32:07.513Z","repository":{"id":197844937,"uuid":"698174643","full_name":"PKU-YuanGroup/Hallucination-Attack","owner":"PKU-YuanGroup","description":"Attack to induce LLMs within hallucinations","archived":false,"fork":false,"pushed_at":"2024-05-17T08:48:15.000Z","size":2860,"stargazers_count":153,"open_issues_count":2,"forks_count":19,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-02T20:22:20.280Z","etag":null,"topics":["adversarial-attacks","ai-safety","deep-learning","hallucinations","llm","llm-safety","machine-learning","nlp"],"latest_commit_sha":null,"homepage":"http://arxiv.org/abs/2310.01469","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PKU-YuanGroup.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-29T10:22:53.000Z","updated_at":"2025-03-27T05:40:49.000Z","dependencies_parsed_at":null,"dependency_job_id":"3910f2d1-466a-47d6-a91b-06965ba9a871","html_url":"https://github.com/PKU-YuanGroup/Hallucination-Attack","commit_stats":null,"previous_names":["pku-yuangroup/hallucination-attack"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PKU-YuanGroup%2FHallucination-Attack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PKU-YuanGroup%2FHallucination-Attack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PKU-YuanGroup%2FHallucination-Attack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PKU-YuanGroup%2FHallucination-Attack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PKU-YuanGroup","download_url":"https://codeload.github.com/PKU-YuanGroup/Hallucination-Attack/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250518007,"owners_count":21443876,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-attacks","ai-safety","deep-learning","hallucinations","llm","llm-safety","machine-learning","nlp"],"created_at":"2024-11-10T09:28:28.597Z","updated_at":"2025-04-23T21:32:04.941Z","avatar_url":"https://github.com/PKU-YuanGroup.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## [LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples](http://arxiv.org/abs/2310.01469)\n\n\u003cdiv align=\"center\"\u003e\n    \u003ca href=\"http://arxiv.org/abs/2310.01469\"\u003e\n        \u003cimg alt=\"arXiv\" src=\"https://img.shields.io/badge/Arxiv-2310.01469-b31b1b.svg?logo=arXiv\" /\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/PKU-YuanGroup/Hallucination-Attack/blob/master/LICENSE\"\u003e\n        \u003cimg alt=\"License\" src=\"https://img.shields.io/badge/Code%20License-MIT-yellow\" /\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://zhuanlan.zhihu.com/p/661444210\"\u003e\n        \u003cimg alt=\"zhihu\" src=\"https://img.shields.io/badge/知乎-0084FF\" /\u003e\n    \u003c/a\u003e\n\u003c/div\u003e\n\n### Brief Intro\nLLMs (e.g., GPT-3.5, LLaMA, and PaLM) suffer from **hallucination**\u0026mdash;fabricating non-existent facts to cheat users without perception.\nAnd the reasons for their existence and pervasiveness remain unclear.\nWe demonstrate that non-sense Out-of-Distribution(OoD) prompts composed of random tokens can also elicit the LLMs to respond with hallucinations.\nThis phenomenon forces us to revisit that **hallucination may be another view of adversarial examples**, and it shares similar features with conventional adversarial examples as the basic feature of LLMs.\nTherefore, we formalize an automatic hallucination triggering method called **hallucination attack** in an adversarial way.\nFollowing is a fake news example generating by hallucination attack.\n\n#### Hallucination Attack generates fake news\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/example-fake.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n#### Weak semantic prompt and OoD prompt can elicit the Vicuna-7B to reply the same fake fact.\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/fig1.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n\n### The Pipeline of Hallucination Attack \nWe substitute tokens via gradient-based token replacing strategy, replacing token reaching smaller negative log-likelihood loss, and induce LLM within hallucinations.\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/fig3.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n### Results on Multiple LLMs\n#### - Vicuna-7B\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/weak-semantic-attack.jpg\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n#### - LLaMA2-7B\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/llama.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n#### - Baichuan-7B-Chat\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/Baichuan2-7B.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n#### - InternLM-7B\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/InternLM-7B.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n### Quick Start\n#### Setup\nYou may config your own base models and their hyper-parameters within `config.py`. Then, you could attack the models or run our demo cases.\n\n#### Demo\nClone this repo and run the code.\n```bash\n$ cd Hallucination-Attack\n```\nInstall the requirements.\n```bash\n$ pip install -r requirements.txt\n```\nRun local demo of hallucination attacked prompt.\n```bash\n$ python demo.py\n```\n\n#### Attack\nStart a new attack training to find a prompt trigger hallucination\n```bash\n$ python main.py\n```\n\n### Citation\n```BibTeX\n@article{yao2023llm,\n  title={LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples},\n  author={Yao, Jia-Yu and Ning, Kun-Peng and Liu, Zhen-Hui and Ning, Mu-Nan and Yuan, Li},\n  journal={arXiv preprint arXiv:2310.01469},\n  year={2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpku-yuangroup%2Fhallucination-attack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpku-yuangroup%2Fhallucination-attack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpku-yuangroup%2Fhallucination-attack/lists"}