awesome-prompt-injection
Learn about a type of vulnerability that specifically targets machine learning models
https://github.com/FonduAI/awesome-prompt-injection
Last synced: 4 days ago
JSON representation
-
Articles and Blog posts
- Prompt injection: What's the worst that can happen? - General overview of Prompt Injection attacks, part of a series.
- ChatGPT Plugins: Data Exfiltration via Images & Cross Plugin Request Forgery - This post shows how a malicious website can take control of a ChatGPT chat session and exfiltrate the history of the conversation.
- Data exfiltration via Indirect Prompt Injection in ChatGPT - This post explores two prompt injections in OpenAI's browsing plugin for ChatGPT. These techniques exploit the input-dependent nature of AI conversational models, allowing an attacker to exfiltrate data through several prompt injection methods, posing significant privacy and security risks.
- Prompt Injection Cheat Sheet: How To Manipulate AI Language Models - A prompt injection cheat sheet for AI bot integrations.
- Prompt injection explained - Video, slides, and a transcript of an introduction to prompt injection and why it's important.
- Adversarial Prompting - A guide on the various types of adversarial prompting and ways to mitigate them.
- Don't you (forget NLP): Prompt injection with control characters in ChatGPT - A look into how to achieve prompt injection from control characters from Dropbox.
- Testing the Limits of Prompt Injection Defence - A practical discussion about the unique complexities of securing LLMs from prompt injection attacks.
- Data exfiltration via Indirect Prompt Injection in ChatGPT - This post explores two prompt injections in OpenAI's browsing plugin for ChatGPT. These techniques exploit the input-dependent nature of AI conversational models, allowing an attacker to exfiltrate data through several prompt injection methods, posing significant privacy and security risks.
- Testing the Limits of Prompt Injection Defence - A practical discussion about the unique complexities of securing LLMs from prompt injection attacks.
- Prompt Injection Cheat Sheet: How To Manipulate AI Language Models - A prompt injection cheat sheet for AI bot integrations.
-
Tutorials
- Prompt Injection - Prompt Injection tutorial from Learn Prompting.
- AI Read Teaming from Google - Google's red team walkthrough of hacking AI systems.
-
Research Papers
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection - This paper explores the concept of Indirect Prompt Injection attacks on Large Language Models (LLMs) through their integration with various applications. It identifies significant security risks, including remote data theft and ecosystem contamination, present in both real-world and synthetic applications.
- Universal and Transferable Adversarial Attacks on Aligned Language Models - This paper introduces a simple and efficient attack method that enables aligned language models to generate objectionable content with high probability, highlighting the need for improved prevention techniques in large language models. The generated adversarial prompts are found to be transferable across various models and interfaces, raising important concerns about controlling objectionable information in such systems.
-
CTF
- Promptalanche - As well as traditional challenges, this CTF also introduce scenarios that mimic agents in real-world applications.
- Gandalf - Your goal is to make Gandalf reveal the secret password for each level. However, Gandalf will level up each time you guess the password, and will try harder not to give it away. Can you beat level 7? (There is a bonus level 8).
- ChatGPT with Browsing is drunk! There is more to it than you might expect at first glance - This riddle requires you to have ChatGPT Plus access and enable the Browsing mode in Settings->Beta Features.
- ChatGPT with Browsing is drunk! There is more to it than you might expect at first glance - This riddle requires you to have ChatGPT Plus access and enable the Browsing mode in Settings->Beta Features.
-
Community
- Learn Prompting - Discord server from Learn Prompting.
-
Tools
- Token Turbulenz - A fuzzer to automate looking for possible Prompt Injections.
Programming Languages
Sub Categories