awesome-MLSecOps
A curated list of MLSecOps tools and resources for securing machine learning and AI systems - adversarial ML defense, LLM security, AI red teaming, model scanning, supply-chain protection, and MLOps pipeline security.
https://github.com/RiccardoBiosas/awesome-MLSecOps
Last synced: 15 days ago
JSON representation
-
101 Resources
-
Activity
-
AI Security Market Map
-
-
AI Security and MLSecOps Beginner Resources
-
Additional Open-Source Tools
-
-
Attack Vectors
- Model Inversion Attacks
- Membership Inference Exploits
- Data Poisoning
- Model Evasion Attacks
- Model Stealing Attacks
- ML Supply Chain Attacks
- Gradient Leakage Attacks
- Cloud Infrastructure Attacks
- Model Denial Of Service
- Architecture and Access-control attacks
- Cloud Infrastructure Attacks
- Model Denial Of Service
-
Blogs and Publications
- Red-Teaming Large Language Models
- Google's AI red-team
- The MLSecOps Top 10 vulnerabilities
- Token Smuggling Jailbreak via Adversarial Prompt
- We need a new way to measure AI security
- PrivacyRaven: Implementing a proof of concept for model inversion
- Adversarial Prompts Engineering
- TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
- Trail Of Bits' audit of Hugging Face's safetensors library
- OWASP Top 10 for Large Language Model Applications
- LLM Security
- Is you MLOps infrastructure leaking secrets?
- Embrace The Red, blog where show how u can hack LLM's.
- Audio-jacking: Using generative AI to distort live audio transactions
- HADESS - Web LLM Attacks
- WTF-blog - MlSecOps frameworks ... Which ones are available and what is the difference?
- DreadNode Paper Stack
- What is MLSecOps
- CircleGuardBench: New Standard for Evaluating AI Moderation Models
- Audio-jacking: Using generative AI to distort live audio transactions
-
Books
- Adversarial AI Attacks, Mitigations, and Defense Strategies: A cybersecurity professional's guide to AI attacks, threat modeling, and securing AI with MLSecOps
- Privacy-Preserving Machine Learning
- Generative AI Security: Theories and Practices (Future of Business and Finance)
- Privacy-Preserving Machine Learning
-
Commercial MLSecOps and AI Security Tools
-
Additional Open-Source Tools
- Cisco AI Defense (formerly Robust Intelligence)
- HiddenLayer AI Detection and Response
- Trent AI - native applications, and code shipped with AI coding tools, traces attack chains, and verifies proposed fixes landed |
- SourceryKit - knowledge proofs, so a call only goes out if its claims check out; hooks the HTTP libraries, logs each outbound call, and blocks endpoints not on the trusted allow-list |
-
-
Commercial Tools
-
Community Resources
-
AI Security Study Map
-
- MLSecOps
- MLSecOps Podcast
- MITRE ATLAS™ (Adversarial Threat Landscape for Artificial-Intelligence Systems)
- OWASP Machine Learning Security Top Ten
- OWASP Top 10 for Large Language Model Applications
- OWASP LLMSVS
- Hackstery
- PWNAI
- AiSec_X_Feed
- HUNTR Discord community
- AI Vulnerability Database
- Incident AI Database
- Defcon AI Village CTF
- Awesome LLM Security
- Awesome AI Security
- MLSecOps Reference Repository
- MlSecOps communtiy
- OWASP AI Exchange
- OWASP Periodic Table of AI Security
- OWASP SLACK
- Awesome LVLM Attack
- Awesome MLLM Safety
- AIRSK
-
-
Contributors ✨
-
Ai Security Market map
- <img src='https://github.com/riccardobiosas.png?size=50'>
- <img src='https://github.com/badarahmed.png?size=50'>
- <img src='https://github.com/deadbits.png?size=50'>
- <img src='https://github.com/wearetyomsmnv.png?size=50'>
- <img src='https://github.com/anmorgan24.png?size=50'>
- <img src='https://github.com/mik0w.png?size=50'>
-
-
DATA
- ARX - Data Anonymization Tool
- Data-Veil
- Tool for IMG anonymization
- Tool for DATA anonymization
- BMW-Anonymization-Api - based training/inference solutions that we already published/will publish for Object Detection and Semantic Segmentation |
- DeepPrivacy2
- PPAP - space-level Image Anonymization with Adversarial Protector Networks|
-
Data Privacy and Anonymization Tools
-
Additional Open-Source Tools
- BMW Anonymization API - based training/inference solutions that we already published/will publish for Object Detection and Semantic Segmentation |
-
-
Infographics
-
AI Security Market Map
- 
Programming Languages
Categories
Threat Modeling
332
Open Source Security Tools
76
MlSecOps pipeline
64
Repositories
34
Community Resources
24
Blogs and Publications
20
Attack Vectors
12
What is MLSecOps?
9
101 Resources
7
MLOps Infrastructure Vulnerabilities
7
DATA
7
Contributors ✨
6
Repository Stats
6
Commercial Tools
5
Books
4
ML Code Security
4
Commercial MLSecOps and AI Security Tools
4
The MLSecOps Hacker Newsletter
4
MLSecOps Pipeline
2
Activity
2
Data Privacy and Anonymization Tools
1
Support Us
1
License
1
Infographics
1
AI Security and MLSecOps Beginner Resources
1
Sub Categories
Keywords
machine-learning
18
security
17
adversarial-attacks
14
llm
12
llm-security
9
deep-learning
7
ai-security
7
pytorch
7
privacy
6
docker
6
prompt-injection
6
python
6
security-tools
6
ai-agents
5
adversarial-examples
5
llm-evaluation
5
tensorflow
5
ai
5
adversarial-example
4
large-language-models
4
llmops
4
supply-chain-security
4
devsecops
4
ai-safety
4
guardrails
4
agent-security
3
natural-language-processing
3
adversarial-machine-learning
3
generative-ai
3
mcp
3
ai-red-team
3
cybersecurity
3
inference
3
model-inversion
3
benchmarking
3
red-teaming
3
owasp
3
keras
3
llm-eval
3
prompt-engineering
3
open-source-security
2
license-compliance
2
ctf
2
firmware-analysis
2
cyclonedx
2
cdxgen
2
awesome
2
golang
2
llm-fuzzer
2
awesome-list
2