https://github.com/thirdkeyai/agentnull
AgentNull: AI System Security Threat Catalog + Proof-of-Concepts. Collection of PoCs for using Agents, MCP, and RAG in bad ways.
https://github.com/thirdkeyai/agentnull
agent agentic-ai agentic-workflow ai blueteam hacks llm llmops mcp mcp-security mcp-server mcp-servers poc proof-of-concept redteam redteam-tools research security-tools threat-intelligence threat-modeling
Last synced: 20 days ago
JSON representation
AgentNull: AI System Security Threat Catalog + Proof-of-Concepts. Collection of PoCs for using Agents, MCP, and RAG in bad ways.
- Host: GitHub
- URL: https://github.com/thirdkeyai/agentnull
- Owner: ThirdKeyAI
- License: mit
- Created: 2025-05-28T21:55:03.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2025-06-06T16:52:23.000Z (10 months ago)
- Last Synced: 2025-10-30T22:39:51.549Z (5 months ago)
- Topics: agent, agentic-ai, agentic-workflow, ai, blueteam, hacks, llm, llmops, mcp, mcp-security, mcp-server, mcp-servers, poc, proof-of-concept, redteam, redteam-tools, research, security-tools, threat-intelligence, threat-modeling
- Language: Python
- Homepage: https://research.thirdkey.ai
- Size: 146 KB
- Stars: 3
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# 🧠 AgentNull: AI System Security Threat Catalog + Proof-of-Concepts
This repository contains a red team-oriented catalog of attack vectors targeting AI systems including autonomous agents (MCP, LangGraph, AutoGPT), RAG pipelines, vector databases, and embedding-based retrieval systems, along with individual proof-of-concepts (PoCs) for each.
## 📘 Structure
- `catalog/AgentNull_Catalog.md` — Human-readable threat catalog
- `catalog/AgentNull_Catalog.json` — Structured version for SOC/SIEM ingestion
- `pocs/` — One directory per attack vector, each with its own README, code, and sample input/output
## ⚠️ Disclaimer
This repository is for **educational and internal security research** purposes only. Do not deploy any techniques or code herein in production or against systems you do not own or have explicit authorization to test.
## 🔧 Usage
Navigate into each `pocs//` folder and follow the README to replicate the attack scenario.
### 🤖 Testing with Local LLMs (Recommended)
For enhanced PoC demonstrations without API costs, use Ollama with local models:
#### Install Ollama
```bash
# Linux/macOS
curl -fsSL https://ollama.ai/install.sh | sh
# Or download from https://ollama.ai/download
```
#### Setup Local Model
```bash
# Pull a lightweight model (recommended for testing)
ollama pull gemma3
# Or use a more capable model
ollama pull deepseek-r1
ollama pull qwen3
```
#### Run PoCs with Local LLM
```bash
# Advanced Tool Poisoning with real LLM
cd pocs/AdvancedToolPoisoning
python3 advanced_tool_poisoning_agent.py local
# Other PoCs work with simulation mode
cd pocs/ContextPackingAttacks
python3 context_packing_agent.py
```
#### Ollama Configuration
- **Default endpoint**: `http://localhost:11434`
- **Model selection**: Edit the model name in PoC files if needed
- **Performance**: Llama2 (~4GB RAM), Mistral (~4GB RAM), CodeLlama (~4GB RAM)
## 🧩 Attack Vectors Covered
### 🤖 MCP & Agent Systems
- **⭐ [Full-Schema Poisoning (FSP)](pocs/FullSchemaPoisoning/)** - Exploit any field in tool schema beyond descriptions
- **⭐ [Advanced Tool Poisoning Attack (ATPA)](pocs/AdvancedToolPoisoning/)** - Manipulate tool outputs to trigger secondary actions
- **⭐ [MCP Rug Pull Attack](pocs/MCPRugPull/)** - Swap benign descriptions for malicious ones after approval
- **⭐ [Schema Validation Bypass](pocs/SchemaValidationBypass/)** - Exploit client validation implementation differences
- **[Tool Confusion Attack](pocs/ToolConfusionAttack/)** - Trick agents into using wrong tools via naming similarity
- **[Nested Function Call Hijack](pocs/NestedFunctionHijack/)** - Use JSON-like data to trigger dangerous function calls
- **[Subprompt Extraction](pocs/SubpromptExtraction/)** - Induce agents to reveal system instructions or tools
- **[Backdoor Planning](pocs/BackdoorPlanning/)** - Inject future intent into multi-step planning for exfiltration
### 🧠 Memory & Context Systems
- **[Recursive Leakage](pocs/RecursiveLeakage/)** - Secrets leak through context summarization
- **[Token Gaslighting](pocs/TokenGaslighting/)** - Push safety instructions out of context via token spam
- **[Heuristic Drift Injection](pocs/HeuristicDriftInjection/)** - Poison agent logic with repeated insecure patterns
- **⭐ [Context Packing Attacks](pocs/ContextPackingAttacks/)** - Overflow context windows to truncate safety instructions
### 🔍 RAG & Vector Systems
- **⭐ [Cross-Embedding Poisoning](pocs/CrossEmbeddingPoisoning/)** - Manipulate embeddings to increase malicious content retrieval
- **⭐ [Index Skew Attacks](pocs/IndexSkewAttacks/)** - Bias vector indices to favor malicious content *(theoretical)*
- **⭐ [Zero-Shot Vector Beaconing](pocs/ZeroShotVectorBeaconing/)** - Embed latent activation patterns for covert signaling *(theoretical)*
- **⭐ [Embedding Feedback Loops](pocs/EmbeddingFeedbackLoops/)** - Poison continual learning systems *(theoretical)*
### 💻 Code & File Systems
- **[Hidden File Exploitation](pocs/HiddenFileExploitation/)** - Get agents to modify `.env`, `.git`, or internal config files
### ⚡ Resource & Performance
- **[Function Flooding](pocs/FunctionFlooding/)** - Generate recursive tool calls to overwhelm budgets/APIs
- **[Semantic DoS](pocs/SemanticDoS/)** - Trigger infinite generation or open-ended tasks
## 📚 Related Research & Attribution
### Novel Attack Vectors (⭐)
The attack vectors marked with ⭐ represent novel concepts primarily developed within the AgentNull project, extending beyond existing documented attack patterns.
### Known Attack Patterns with Research Links
- **Recursive Leakage**: [Lost in the Middle: How Language Models Use Long Contexts](https://arxiv.org/abs/2307.03172)
- **Heuristic Drift Injection**: [Poisoning Web-Scale Training Data is Practical](https://arxiv.org/abs/2302.10149)
- **Tool Confusion Attack**: [LLM-as-a-judge](https://arxiv.org/abs/2306.05685)
- **Token Gaslighting**: [RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture](https://arxiv.org/abs/2401.08406)
- **Function Flooding**: [Denial-of-Service Attack on Test-Time-Tuning Models](https://arxiv.org/abs/2405.02324)
- **Hidden File Exploitation**: [OWASP Top 10 for Large Language Model Applications](https://owasp.org/www-project-top-10-for-large-language-model-applications/)
- **Backdoor Planning**: [Backdoor Attacks on Language Models](https://arxiv.org/abs/2311.09403)
- **Nested Function Call Hijack**: [OWASP Top 10 for Large Language Model Applications](https://owasp.org/www-project-top-10-for-large-language-model-applications/)
### Sponsored by [ThirdKey](https://thirdkey.ai)