Projects in Awesome Lists tagged with llm-security
A curated list of projects in awesome lists tagged with llm-security .
https://github.com/pathwaycom/llm-app
Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.
chatbot hugging-face llm llm-local llm-prompting llm-security llmops machine-learning open-ai pathway rag real-time retrieval-augmented-generation vector-database vector-index
Last synced: 12 May 2025
https://github.com/giskard-ai/giskard
🐢 Open-Source Evaluation & Testing for AI & LLM systems
agent-evaluation ai-red-team ai-security ai-testing fairness-ai llm llm-eval llm-evaluation llm-security llmops ml-testing ml-validation mlops rag-evaluation red-team-tools responsible-ai trustworthy-ai
Last synced: 14 May 2025
https://github.com/Giskard-AI/giskard
🐢 Open-Source Evaluation & Testing for AI & LLM systems
agent-evaluation ai-red-team ai-security ai-testing fairness-ai llm llm-eval llm-evaluation llm-security llmops ml-testing ml-validation mlops rag-evaluation red-team-tools responsible-ai trustworthy-ai
Last synced: 15 Apr 2025
https://github.com/NVIDIA/garak
the LLM vulnerability scanner
ai llm-evaluation llm-security security-scanners vulnerability-assessment
Last synced: 13 May 2025
https://github.com/nvidia/garak
the LLM vulnerability scanner
ai llm-evaluation llm-security security-scanners vulnerability-assessment
Last synced: 11 May 2025
https://github.com/verazuo/jailbreak_llms
[CCS'24] A dataset consists of 15,140 ChatGPT prompts from Reddit, Discord, websites, and open-source datasets (including 1,405 jailbreak prompts).
chatgpt jailbreak large-language-model llm llm-security prompt
Last synced: 25 Mar 2025
https://github.com/protectai/llm-guard
The Security Toolkit for LLM Interactions
adversarial-machine-learning chatgpt large-language-models llm llm-security llmops prompt-engineering prompt-injection security-tools transformers
Last synced: 13 May 2025
https://github.com/mariocandela/beelzebub
A secure low code honeypot framework, leveraging LLM for System Virtualization.
cloudnative cloudsecurity cybersecurity deception framework go golang honeypot kubernetes llama3 llm llm-honeypot llm-security low-code ollama openai research research-project security whitehat
Last synced: 15 May 2025
https://github.com/deadbits/vigil-llm
⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs
adversarial-attacks adversarial-machine-learning large-language-models llm-security llmops prompt-injection security-tools yara-scanner
Last synced: 06 Apr 2025
https://github.com/splx-ai/agentic-radar
A security scanner for your LLM agentic workflows
agentic-ai agentic-framework agentic-workflow ai ai-red-teaming ai-security cli devsecops generative-ai llm llm-security red-teaming security security-tools
Last synced: 01 Apr 2025
https://github.com/r3drun3/sploitcraft
🏴☠️ Hacking Guides, Demos and Proof-of-Concepts 🥷
ai aws cloud container-security cybersecurity docker hacking hacking-tutorials linux llm-security network-security offensive-security proof-of-concept python redteam tutorials web-vulnerabilities windows
Last synced: 04 Apr 2025
https://github.com/liu00222/Open-Prompt-Injection
This repository provides implementation to formalize and benchmark Prompt Injection attacks and defenses
llm llm-security llms prompt-injection prompt-injection-tool security-and-privacy
Last synced: 21 Jan 2025
https://github.com/yevh/taac-ai
AI-driven Threat modeling-as-a-Code (TaaC-AI)
ai application-security claude-3 devsecops gpt gpt-3 gpt-4 llm-security mistral-7b secure-development taac threat threat-modeling threat-modeling-from-code threat-modeling-tool threat-models
Last synced: 03 Apr 2025
https://github.com/ZenGuard-AI/fast-llm-security-guardrails
The fastest && easiest LLM security guardrails for AI Agents and applications.
adversarial-machine-learning llm-guard llm-guardrails llm-privacy llm-security prompt-security security
Last synced: 06 Jan 2025
https://github.com/LLAMATOR-Core/llamator
Framework for testing vulnerabilities of large language models (LLM).
agent ai ai-security attack hallucinations jailbreak llm llm-read-team llm-security llm-testing misinformation nlp owasp python rag rag-evaluation red-team red-team-tools security-tools vulnerability
Last synced: 10 May 2025
https://github.com/lakeraai/pint-benchmark
A benchmark for prompt injection detection systems.
benchmark llm llm-benchmarking llm-security prompt-injection
Last synced: 06 Apr 2025
https://github.com/raga-ai-hub/raga-llm-hub
Framework for LLM evaluation, guardrails and security
guardrails llm-evaluation llm-security llmops
Last synced: 23 Apr 2025
https://github.com/llamator-core/llamator
Framework for testing vulnerabilities of large language models (LLM).
ai ai-security attack hallucinations jailbreak llm llm-read-team llm-security llm-testing misinformation nlp owasp python rag rag-evaluation red-team red-team-tools red-teaming security-tools vulnerability-assessment
Last synced: 07 Apr 2025
https://github.com/arekusandr/last_layer
Ultra-fast, low latency LLM prompt injection/jailbreak detection ⛓️
chatgpt-prompts jailbreak large-language-models llm-guard llm-guardrails llm-local llm-security prompt-engineering security-tools
Last synced: 13 Jan 2025
https://github.com/romiconez/llamator
Framework for testing vulnerabilities of large language models (LLM).
ai ai-security attack hallucinations jailbreak llm llm-read-team llm-security llm-testing misinformation nlp owasp python rag rag-evaluation red-team red-team-tools red-teaming security-tools vulnerability-assessment
Last synced: 22 Mar 2025
https://github.com/pdparchitect/llm-hacking-database
This repository contains various attack against Large Language Models.
hacking llm llm-security security
Last synced: 29 Dec 2024
https://github.com/llm-platform-security/SecGPT
An Execution Isolation Architecture for LLM-Based Agentic Systems
chatgpt gpt isolation langchain llm llm-agent llm-based-systems llm-framework llm-platform llm-privacy llm-security multi-agent-systems openai-api sandbox
Last synced: 24 Mar 2025
https://github.com/RomiconEZ/LLaMator
Framework for testing vulnerabilities of large language models (LLM).
ai ai-security attack framework hallucinations jailbreaks llm llm-red-team llm-security llm-testing misinformation nlp python rag rag-evaluation red-team red-team-framework red-teaming security-tools vulnerability-assessment
Last synced: 29 Dec 2024
https://github.com/leondz/lm_risk_cards
Risks and targets for assessing LLMs & LLM vulnerabilities
llm llm-security red-teaming security vulnerability
Last synced: 10 Apr 2025
https://github.com/sinanw/llm-security-prompt-injection
This project investigates the security of large language models by performing binary classification of a set of input prompts to discover malicious prompts. Several approaches have been analyzed using classical ML algorithms, a trained LLM model, and a fine-tuned LLM model.
cybersecurity llm-prompting llm-security prompt-injection transformers-models
Last synced: 26 Nov 2024
https://github.com/SEC-CAFE/handbook
安全手册，企业安全实践、攻防与安全研究知识库
ai-security awesome-security llm-security security-handbook security-wiki
Last synced: 16 Feb 2025
https://github.com/lakeraai/chainguard
Guard your LangChain applications against prompt injection with Lakera ChainGuard.
langchain langchain-python llm llm-security prompt-injection
Last synced: 19 Apr 2025
https://github.com/sap/stars
AI agent whose purpose is to conduct vulnerability tests on LLMs from SAP AI Core or from local deployments, or models from HuggingFace. The goal of this project is to identify and correct any potential security vulnerabilities.
ai ai-agents ai-security llm llm-security security
Last synced: 13 Apr 2025
https://github.com/msoedov/agentic_security
Agentic LLM Vulnerability Scanner
llm-guardrails llm-jailbreaks llm-scanner llm-security llm-vulnerabilities owasp-llm-top-10
Last synced: 01 Jan 2025
https://github.com/levitation-opensource/manipulative-expression-recognition
MER is a software that identifies and highlights manipulative communication in text from human conversations and AI-generated responses. MER benchmarks language models for manipulative expressions, fostering development of transparency and safety in AI. It also supports manipulation victims by detecting manipulative patterns in human communication.
benchmarking conversation-analysis conversation-analytics expression-recognition fraud-detection fraud-prevention human-computer-interaction human-robot-interaction llm llm-security llm-test llm-training manipulation misinformation prompt-engineering prompt-injection psychometrics sentiment-analysis sentiment-classification transparency
Last synced: 16 Mar 2025
https://github.com/balavenkatesh3322/guardrails-demo
LLM Security Project with Llama Guard
aisecurity attack-defense generative-ai llama-2 llama-guard llm llm-security llmops prompt-injection-tool security
Last synced: 12 Apr 2025
https://github.com/aintrust-ai/aixploit
Engineered to help red teams and penetration testers exploit large language model AI solutions vulnerabilities.
adversarial-attacks adversarial-machine-learning chatgpt hacking large-language-models llm llm-guardrails llm-security prompt-injection redteaming
Last synced: 11 Apr 2025
https://github.com/dwain-barnes/purpaas-llm
PurPaaS is an innovative open-source security testing platform that implements purple teaming (combined red and blue team approaches) to evaluate local LLM models through Ollama. By orchestrating autonomous agents, PurPaaS provides comprehensive security assessment of locally deployed AI models.
ai-safety ai-security ai-vulnerability-assessment autonomous-agents blue-team cyber-security llm-security llms local-llm ollama open-source purple-teaming red-team security-testing
Last synced: 05 May 2025
https://github.com/dewitt4/ai-security-alerts
Security monitoring system that logs suspicious activities and alerts your security team, allowing you to make informed decisions about escalating genuine threats.
ai ai-sec ai-security cybersecurity llm-security
Last synced: 12 Apr 2025
https://github.com/dewitt4/llmguardian
Comprehensive LLM AI Model protection | Protect your production GenAI LLM applications | cybersecurity toolset aligned to addressing OWASP vulnerabilities in Large Language Models - https://genai.owasp.org/llm-top-10/
ai ai-sec ai-security ai-security-tool llm-security
Last synced: 12 Apr 2025
https://github.com/0x6f677548/copilot-instructions-unicode-injection
Proof of Concept (PoC) demonstrating prompt injection vulnerability in AI code assistants (like Copilot) using hidden Unicode characters within instruction files (copilot-instructions.md). Highlights risks of using untrusted instruction templates. For educational/research purposes only.
ai-security copilot llm-security prompt-injection prompt-injection-llm-security red-teaming security-research unicode
Last synced: 05 May 2025
https://github.com/drlordbasil/ultimate-jail-break-conversion-prompter
Jailbreak common AI censors. Working on this for educational purposes and to earn some money from bounties.
ai jailbreak llm llm-security prompt prompting
Last synced: 25 Jan 2025
https://github.com/nagababumo/red-teaming-llm-applications
giskard jailbreak llm-security prompt-injection red-teaming
Last synced: 03 Mar 2025
https://github.com/nodite/llm-guard-ts
The Security Toolkit for LLM Interactions (TS version)
adversarial-machine-learning chatgpt large-language-models llm llm-security llmops prompt-engineering prompt-injection security-tools transformers typescript
Last synced: 28 Mar 2025
https://github.com/dwain-barnes/owasp-top10-llm-playbook-generator
An interactive web application that generates comprehensive security playbooks for mitigating the OWASP Top 10 vulnerabilities specific to Large Language Model (LLM) applications. The application consists of a Flask backend that leverages the OpenAI API to generate detailed playbooks, paired with a simple HTML/JavaScript frontend.
cybersecurity generative-ai large-language-models llm-security openai-api owasp owasp-llm owasp-top-10 security-playbooks
Last synced: 21 Mar 2025