{"id":13677966,"url":"https://github.com/prompt-security/ps-fuzz","last_synced_at":"2026-01-14T08:40:29.796Z","repository":{"id":233637498,"uuid":"735087691","full_name":"prompt-security/ps-fuzz","owner":"prompt-security","description":"Make your GenAI Apps Safe \u0026 Secure :rocket: Test \u0026 harden your system prompt","archived":false,"fork":false,"pushed_at":"2025-09-23T13:50:52.000Z","size":19697,"stargazers_count":566,"open_issues_count":18,"forks_count":81,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-09-25T11:20:54.480Z","etag":null,"topics":["ai","ai-fuzzer","fuzzer","generative-ai","llm","llm-fuzzer","security","security-tools","system-prompt-hardener"],"latest_commit_sha":null,"homepage":"https://www.prompt.security/fuzzer","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prompt-security.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-12-23T16:09:31.000Z","updated_at":"2025-09-24T21:16:12.000Z","dependencies_parsed_at":"2024-04-20T02:28:05.907Z","dependency_job_id":"8433ad3a-247a-4c3e-93fc-7b4380e30589","html_url":"https://github.com/prompt-security/ps-fuzz","commit_stats":null,"previous_names":["prompt-security/ps-fuzz"],"tags_count":18,"template":false,"template_full_name":null,"purl":"pkg:github/prompt-security/ps-fuzz","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prompt-security%2Fps-fuzz","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prompt-security%2Fps-fuzz/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prompt-security%2Fps-fuzz/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prompt-security%2Fps-fuzz/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prompt-security","download_url":"https://codeload.github.com/prompt-security/ps-fuzz/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prompt-security%2Fps-fuzz/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28414676,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:38:59.149Z","status":"ssl_error","status_checked_at":"2026-01-14T08:38:43.588Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-fuzzer","fuzzer","generative-ai","llm","llm-fuzzer","security","security-tools","system-prompt-hardener"],"created_at":"2024-08-02T13:00:48.936Z","updated_at":"2026-01-14T08:40:29.773Z","avatar_url":"https://github.com/prompt-security.png","language":"Python","funding_links":[],"categories":["Python","Open Source Security Tools","Agent Security","Tools","Attack Techniques \u0026 Red Teaming","⚔️ LLM And GenAI Security Testing Tools","AI Red Teaming (Testing AI Targets)"],"sub_categories":["Survey","LLM \u0026 GenAI Red Teaming"],"readme":"\u003ch1 align=\"center\"\u003e\n  \u003cimg src=\"resources/prompt-icon.svg\" alt=\"prompt-icon\"\u003e\n  Prompt Fuzzer\n  \u003cimg src=\"resources/prompt-icon.svg\" alt=\"prompt-icon\"\u003e\n\u003c/h1\u003e\n\n\u003ch2 align=\"center\"\u003e\n  The open-source tool to help you harden your GenAI applications\n\u003cbr\u003e\n\u003cbr\u003e\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n![ci](https://github.com/prompt-security/ps-fuzz/actions/workflows/ci.yml/badge.svg)\n![GitHub contributors](https://img.shields.io/github/contributors/prompt-security/ps-fuzz)\n![Last release](https://img.shields.io/github/v/release/prompt-security/ps-fuzz)\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/148n5M1wZXp-ojhnh-_KP01OYtUwJwlUl?usp=sharing)\n\u003c/h2\u003e\n\n\n\u003cdiv align=\"center\"\u003e\n\n\u003ch4\u003e Brought to you by Prompt Security, the Complete Platform for GenAI Security\n\n\u003c/div\u003e\n\n---\n\n\u003cdiv align=\"center\"\u003e\n  \n![Prompt Security Logo](./resources/Black+Color.png)\n\n\u003c/div\u003e\n\n---\n\n\nTable of Contents\n-----------------\n\n\u003c!-- vim-markdown-toc GFM --\u003e\n* [ :sparkles: About](#what-is-prompt-fuzzer)\n* [ :rotating_light: Features](#features)\n* [ :rocket: Installation](#installation)\n    * [Using pip](#using-pip)\n    * [Package page](https://pypi.org/project/prompt-security-fuzzer/)\n    * [:construction: Using docker](#docker) ***coming soon*** \n* [Usage](#usage)\n    * [Features](#features)\n    * [Environment variables](#environment-variables)\n    * [Supported LLMs](#llm-providers)\n    * [Command line options](#options)\n* [Examples](#examples)\n    * [Interactive mode](#interactive)\n    * [Quickstart single run](#singlerun)\n* [ :clapper: Demo video](#demovideo)\n* [Supported attacks](#attacks)\n   * [Jailbreak](#jailbreak)\n   * [Prompt Injection](#pi-injection)\n   * [System prompt extraction](#systemleak)\n* [ :rainbow:  What’s next on the roadmap?](#roadmap)\n* [ :beers: Contributing](#contributing)\n\n\u003cbr\u003e\n\n\n\u003ca id=\"what-is-prompt-fuzzer\"\u003e\u003c/a\u003e\n\n\n## ✨ What is the Prompt Fuzzer\n1. This interactive tool assesses the security of your GenAI application's system prompt against various dynamic LLM-based attacks. It provides a security evaluation based on the outcome of these attack simulations, enabling you to strengthen your system prompt as needed.\n2. The Prompt Fuzzer dynamically tailors its tests to your application's unique configuration and domain.\n3. The Fuzzer also includes a Playground chat interface, giving you the chance to iteratively improve your system prompt, hardening it against a wide spectrum of generative AI attacks.\n\n:warning: Using the Prompt Fuzzer will lead to the consumption of tokens. :warning:\n\n\u003cbr\u003e\n\n\u003ca id=\"installation\"\u003e\u003c/a\u003e\n## 🚀 Installation \n![prompt-fuzzer-install-final](https://github.com/prompt-security/ps-fuzz/assets/163823698/47daaeed-3fad-417e-b646-06753db427f4)\n\n1. Install the Fuzzer package\n   \u003ca id=\"using-pip\"\u003e\u003c/a\u003e\n   #### Using pip install\n   ```zsh\n   pip install prompt-security-fuzzer\n   ```\n   \u003ca id=\"using-pypi\"\u003e\u003c/a\u003e\n   #### Using the package page on PyPi\n   You can also visit the [package page](https://pypi.org/project/prompt-security-fuzzer/) on PyPi\n\n   Or grab latest release wheel file form [releases](https://github.com/prompt-security/ps-fuzz/releases)\n\n2. Launch the Fuzzer\n   ```zsh\n   export OPENAI_API_KEY=sk-123XXXXXXXXXXXX\n   \n   prompt-security-fuzzer\n   ```\n\n3. Input your system prompt\n\n4. Start testing\n\n5. Test yourself with the Playground! Iterate as many times are you like until your system prompt is secure.\n\n\n\u003ca id=\"usage\"\u003e\u003c/a\u003e\n## :computer:  Usage\n\u003ca id=\"features\"\u003e\u003c/a\u003e\n### Features\n\u003cb\u003eThe Prompt Fuzzer Supports:\u003c/b\u003e\u003cbr\u003e\n🧞  16 [llm providers](#llm-providers)\u003cbr\u003e\n🔫  15 different [attacks](#attacks)\u003cbr\u003e\n💬  Interactive mode\u003cbr\u003e\n🤖  CLI mode\u003cbr\u003e\n🧵  Multi threaded testing\u003cbr\u003e\n  \n\u003ca id=\"environment-variables\"\u003e\u003c/a\u003e\n### Environment variables:\n\nYou need to set an environment variable to hold the access key of your preferred LLM provider.\ndefault is  `OPENAI_API_KEY`\n\nExample: set `OPENAI_API_KEY` with your API Token to use with your OpenAI account.\n\nAlternatively, create a file named `.env` in the current directory and set the `OPENAI_API_KEY` there.\n\u003ca id=\"llm-providers\"\u003e\u003c/a\u003e\n\n\u003cdetails\u003e\u003csummary\u003eWe're fully LLM agnostic. (Click for full configuration list of llm providers)\u003c/summary\u003e\n\n| ENVIORMENT KEY| Description |\n|---------------|-------------|\n| `ANTHROPIC_API_KEY` | `Anthropic` Chat large language models.|\n| `ANYSCALE_API_KEY` |  `Anyscale` Chat large language models.|\n| `AZURE OPENAI_API_KEY` | `Azure OpenAI` Chat Completion API.|\n| `BAICHUAN_API_KEY` |  `Baichuan chat` models API by Baichuan Intelligent Technology.|\n| `COHERE_API_KEY` | `Cohere chat` large language models.|\n| `EVERLYAI_API_KEY` | `EverlyAI` Chat large language models|\n| `FIREWORKS_API_KEY` | `Fireworks` Chat models|\n| `GIGACHAT_CREDENTIALS` |  `GigaChat` large language models API. |\n| `GOOGLE_API_KEY` |  `Google PaLM` Chat models API.|\n| `JINA_API_TOKEN` |  `Jina AI` Chat models API.|\n| `KONKO_API_KEY` | `ChatKonko` Chat large language models API.|\n| `MINIMAX_API_KEY`, `MINIMAX_GROUP_ID` | Wrapper around Minimax large language models.|\n| `OPENAI_API_KEY` | `OpenAI` Chat large language models API.|\n| `PROMPTLAYER_API_KEY` |  `PromptLayer` and OpenAI Chat large language models API.|\n| `QIANFAN_AK`, `QIANFAN_SK` |  `Baidu Qianfan` chat models.|\n| `YC_API_KEY` | `YandexGPT` large language models.|\n\u003c/details\u003e\n\n\u003cbr/\u003e\n\u003cbr/\u003e\n\n\u003ca id=\"options\"\u003e\u003c/a\u003e\n### Command line Options\n* `--list-providers`        Lists all available providers\n* `--list-attacks`          Lists available attacks and exit\n* `--attack-provider`       Attack Provider \n* `--attack-model`          Attack Model  \n* `--target-provider `      Target provider\n* `--target-model`          Target model  \n* `--num-attempts, -n`       NUM_ATTEMPTS Number of different attack prompts \n* `--num-threads, -t`        NUM_THREADS  Number of worker threads \n* `--attack-temperature, -a` ATTACK_TEMPERATURE  Temperature for attack model \n* `--debug-level, -d`        DEBUG_LEVEL  Debug level (0-2)   \n* `-batch, -b`               Run the fuzzer in unattended (batch) mode, bypassing the interactive steps \n\n\u003cbr/\u003e\n\n\u003ca id=\"examples\"\u003e\u003c/a\u003e\n## Examples\n\nSystem prompt examples (of various strengths) can be found in the subdirectory [system_prompt.examples](https://github.com/prompt-security/ps-fuzz/tree/main/system_prompt.examples/) in the sources.\n \n\u003ca id=\"interactive\"\u003e\u003c/a\u003e\n#### Interactive mode (default mode)\n\n  Run tests against the system prompt\n\n```\n    prompt_security_fuzzer \n```\n\n\u003ca id=\"singlerun\"\u003e\u003c/a\u003e\n#### :speedboat:  Quick start single run\n\nRun tests against the system prompt (in non-interactive batch mode):\n\n```\n    prompt-security-fuzzer -b ./system_prompt.examples/medium_system_prompt.txt\n```\n\n#### 📺 Custom Benchmark!\nRun tests against the system prompt with a custom benchmark\n\n```\n    prompt-security-fuzzer -b ./system_prompt.examples/medium_system_prompt.txt --custom-benchmark=ps_fuzz/attack_data/custom_benchmark1.csv\n```\n\n#### 🐹 Run only a subset of attacks!\nRun tests against the system prompt with a subset of attacks\n\n```\n    prompt-security-fuzzer -b ./system_prompt.examples/medium_system_prompt.txt --custom-benchmark=ps_fuzz/attack_data/custom_benchmark1.csv --tests='[\"ucar\",\"amnesia\"]'\n```\n\n\u003cbr\u003e\n\u003cbr\u003e\n\u003cbr\u003e\n\n\u003ca id=\"colab\"\u003e\u003c/a\u003e\n## 📓 Google Colab Notebook\nRefine and harden your system prompt in our [Google Colab Notebook](https://colab.research.google.com/drive/148n5M1wZXp-ojhnh-_KP01OYtUwJwlUl?usp=sharing)\u003cbr\u003e\u003cbr\u003e\n\u003cimg src=\"./resources/PromptFuzzer.png\" alt=\"Prompt Fuzzer Refinement Process\"/\u003e\n\u003cbr\u003e\u003cbr\u003e\n\u003ca id=\"demovideo\"\u003e\u003c/a\u003e\n## 🎬 Demo video\n[![Watch the video](https://img.youtube.com/vi/xCO3_LHPH2Q/hqdefault.jpg)](https://www.youtube.com/watch?v=xCO3_LHPH2Q)\n\n\u003ca id=\"attacks\"\u003e\u003c/a\u003e\n## :crossed_swords: Simulated Attack Details\n\nWe use a dynamic testing approach, where we get the necessary context from your System Prompt and based on that adapt the fuzzing process.\n\n\u003ca id=\"jailbreak\"\u003e\u003c/a\u003e\n##### Jailbreak\n \n- **AIM Jailbreak**: Probes ethical compliance by eliciting instructions on illegal activities via Machiavelli roleplay.\n- **Affirmative Suffix**: Exploits a vulnerability to elicit affirmative responses for prohibited content.\n- **Amnesia**: Causes the LLM to forget or disregard system prompt instructions, leading to unrelated outputs.\n- **Contextual Redirection**: Tricks the LLM into adopting a specific role that may bypass preset restrictions.\n- **Do Anything Now Jailbreak (DAN)**: Tests LLM resilience against performing forbidden activities through roleplay.\n- **Harmful Behavior**: Challenges the LLM with prompts from a harmful behavior dataset.\n- **Linguistic Evasion**: Uses non-English languages to bypass preset restrictions or focus.\n- **Self Refine**: Attempts to bypass system prompts by refining responses towards unethical directions.\n- **UCAR**: Assesses LLM adherence to content restrictions when asked to ignore such filters.\n- **Base64 Evasion** Attempts to evade detection of harmful prompts by encoding them into base64 strings\n\n\u003ca id=\"pi-injection\"\u003e\u003c/a\u003e\n##### Prompt Injection\n\n- **Authoritative Role Impersonation**: Impersonates an authoritative role to misdirect the LLM's outputs.\n- **Complimentary Transition**: Tests ability to maintain content standards when switching topics after a compliment.\n- **Ethical Compliance**: Evaluates resistance to discussing harmful or inappropriate content about sensitive topics.\n- **Typoglycemia Attack**: Exploits text processing vulnerabilities by omitting random characters, causing incorrect responses.\n\n\u003ca id=\"systemleak\"\u003e\u003c/a\u003e\n##### System prompt extraction\n\n- **System Prompt Stealer**: Attempts to extract the LLM's internal configuration or sensitive information.\n\n##### Definitions\n\n- **Broken**: Attack type attempts that LLM succumbed to.\n- **Resilient**: Attack type attempts that LLM resisted.\n- **Errors**: Attack type attempts that had inconclusive results.\n\n\n\n\u003cbr/\u003e\n\u003cbr/\u003e\n\n\u003ca id=\"roadmap\"\u003e\u003c/a\u003e\n## :rainbow: What’s next on the roadmap?\n\n- [X]  Google Colab Notebook\n- [X]  Adjust the output evaluation mechanism for prompt dataset testing\n- [ ]  Continue adding new GenAI attack types\n- [ ]  Enhaced reporting capabilites\n- [ ]  Hardening recommendations\n\nTurn this into a community project! We want this to be useful to everyone building GenAI applications. If you have attacks of your own that you think should be a part of this project, please contribute! This is how: https://github.com/prompt-security/ps-fuzz/blob/main/CONTRIBUTING.md\n\n\u003ca id=\"contributing\"\u003e\u003c/a\u003e\n## 🍻 Contributing\n\nInterested in contributing to the development of our tools? Great! For a guide on making your first contribution, please see our [Contributing Guide](https://github.com/prompt-security/ps-fuzz/blob/main/CONTRIBUTING.md#get-started-with-your-first-contribution-adding-a-new-test). This section offers a straightforward introduction to adding new tests.\n\nFor ideas on what tests to add, check out the issues tab in our GitHub repository. Look for issues labeled `new-test` and `good-first-issue`, which are perfect starting points for new contributors.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprompt-security%2Fps-fuzz","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprompt-security%2Fps-fuzz","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprompt-security%2Fps-fuzz/lists"}