awesome-llm-security
A curated list of tools, frameworks, and resources for securing LLM applications, agents, and AI infrastructure.
https://github.com/devgreick/awesome-llm-security
Last synced: 1 day ago
JSON representation
-
Agent & MCP Security
-
Agent Scanners
- Aegis - sagwekar2652/aegis) |
- Agentic Radar - ai/agentic-radar) |
-
MCP Security
- MCP Guardian - guardian) |
- MCP Security Checklist - Security-Checklist) |
- Secure MCP Gateway - mcp-gateway) |
-
Sandboxing & Isolation
- E2B - dev/e2b) |
- KubeStellar Console - cluster Kubernetes dashboard with Kyverno policy enforcement, RBAC visualization, supply chain security, and AI/LLM workload access controls. CNCF Sandbox project | ๐ก | ๐ | [GitHub](https://github.com/kubestellar/console) |
- ShellWard - layer defense, zero deps | ๐ก | ๐ | [GitHub](https://github.com/nicholasgcoles/shellward) |
-
-
Commercial Platforms
-
Pentesting Assistants
- Calypso AI - time AI security platform with policy enforcement and monitoring | ๐ข | ๐ฐ | [Website](https://calypsoai.com/) |
- HiddenLayer
- Lakera - time AI security API with prompt injection firewall | ๐ข | ๐ฐ | [Website](https://www.lakera.ai/) |
- Lasso Security
- Prompt Security
- Robust Intelligence
-
-
Frameworks & Standards
-
Pentesting Assistants
- EU AI Act
- MITRE ATLAS
- NIST AI RMF - intelligence) |
- OWASP Top 10 for Agentic AI 2026 - ASI10) | [Website](https://genai.owasp.org/) |
-
-
Guardrails & Defense
-
Input/Output Scanners
- APort Agent Guardrails - action authorization guardrails for AI agents and MCP/tool-use workflows. | Runtime authorization guardrails | Runtime authorization guardrails | https://aport.io |
- LLM Guard - guard) |
- Rebuff - hardening prompt injection detector with canary tokens | ๐ก | ๐ | [GitHub](https://github.com/protectai/rebuff) |
- TrustGate - ai/trustgate) |
- Vigil - llm) |
-
PII & Data Leakage Prevention
-
Programmable Guardrails
- Guardrails AI - ai/guardrails) |
- NeMo Guardrails - Guardrails) |
-
-
Learning & CTF
-
Pentesting Assistants
- 0din GenAI Bug Bounty
- Damn Vulnerable LLM App - vulnerable-llm-app) |
- Gandalf
- HackAPrompt - 2023) |
- LLMSecurityGuide
- Prompt Airlines - on GenAI security challenges | ๐ | [Website](https://promptairlines.com/) |
-
-
LLM-Powered Security Tools
-
Pentesting Assistants
- PentestGPT - powered automated penetration testing tool | ๐กโญ | ๐ | [GitHub](https://github.com/GreyDGL/PentestGPT) |
-
Reverse Engineering
-
Vulnerability Detection
- ai for Pwndbg - ai) |
- HackingBuddyGPT - lab/hackingBuddyGPT) |
-
-
Model Security
-
Adversarial Robustness
- ART - AI/adversarial-robustness-toolbox) |
- Counterfit
- Foolbox
- TextAttack
-
Model Scanning
-
Watermarking & Detection
- GPTZero
- MarkMyWords - c/MarkMyWords) |
-
-
RAG Security
-
Watermarking & Detection
- LangKit
- Plexiglass - 2024/plexiglass) |
-
-
Red Teaming & Offensive
-
Benchmarks & Datasets
- AgentDojo - spylab/agentdojo) |
- CyberSecEval - llama/PurpleLlama) |
- HarmBench
-
Jailbreak Frameworks
- EasyJailbreak
- FuzzyAI
- GPTFuzzer - generated jailbreak prompts via fuzzing | ๐ข | ๐ | [GitHub](https://github.com/sherdencooper/GPTFuzz) |
- JailbreakBench
- Purple Llama - llama/PurpleLlama) |
-
LLM Vulnerability Scanners
-
Prompt Injection Testing
- HouYi - integrated apps | ๐ก | ๐ | [GitHub](https://github.com/LLMSecurity/HouYi) |
- LLMFuzzer
- Open-Prompt-Injection - Prompt-Injection) |
- Promptmap
- ps-fuzz - security/ps-fuzz) |
-
Programming Languages
Categories
Sub Categories
Pentesting Assistants
17
LLM Vulnerability Scanners
5
Jailbreak Frameworks
5
Prompt Injection Testing
5
Input/Output Scanners
5
Adversarial Robustness
4
Watermarking & Detection
4
Sandboxing & Isolation
3
Benchmarks & Datasets
3
MCP Security
3
PII & Data Leakage Prevention
2
Programmable Guardrails
2
Reverse Engineering
2
Vulnerability Detection
2
Model Scanning
2
Agent Scanners
2
Keywords
llm
10
prompt-injection
7
llm-security
7
ai
6
security
6
large-language-models
5
python
5
machine-learning
4
ai-security
4
adversarial-attacks
4
nlp
3
prompt-engineering
3
llmops
3
generative-ai
3
adversarial-examples
3
adversarial-machine-learning
3
security-tools
3
privacy
2
jailbreak
2
trustworthy-ai
2
llms
2
ai-red-team
2
agentic-ai
2
red-team-tools
2
red-teaming
2
responsible-ai
2
llm-evaluation
2
penetration-testing
2
data-augmentation
1
observability
1
nlg
1
generative-ai-security-assurance
1
natural-language-processing
1
anonymization
1
data-anonymization
1
data-masking
1
data-obfuscation
1
data-privacy
1
data-redaction
1
de-identification
1
guardrails
1
image-redactor
1
named-entity-recognition
1
personally-identifiable-information
1
prompt-injection-llm-security
1
prompt-injection-defense
1
offensive-security
1
gpt-35-turbo
1
gpt-4-turbo
1
ida-pro
1