awesome-MLSecOps

A curated list of MLSecOps tools and resources for securing machine learning and AI systems - adversarial ML defense, LLM security, AI red teaming, model scanning, supply-chain protection, and MLOps pipeline security.
https://github.com/RiccardoBiosas/awesome-MLSecOps

Last synced: 8 days ago
JSON representation

101 Resources

Activity

AI Security Market Map

AI Security and MLSecOps Beginner Resources

Additional Open-Source Tools

Microsoft - Generative AI Security for Beginners

Attack Vectors

Blogs and Publications

Books

Commercial MLSecOps and AI Security Tools

Additional Open-Source Tools

Cisco AI Defense (formerly Robust Intelligence)
HiddenLayer AI Detection and Response
Trent AI - native applications, and code shipped with AI coding tools, traces attack chains, and verifies proposed fixes landed |
SourceryKit - knowledge proofs, so a call only goes out if its claims check out; hooks the HTTP libraries, logs each outbound call, and blocks endpoints not on the trusted allow-list |

Commercial Tools

Contributors ✨

Ai Security Market map

DATA

ARX - Data Anonymization Tool
Data-Veil
Tool for IMG anonymization
Tool for DATA anonymization
BMW-Anonymization-Api - based training/inference solutions that we already published/will publish for Object Detection and Semantic Segmentation |
DeepPrivacy2
PPAP - space-level Image Anonymization with Adversarial Protector Networks|

Data Privacy and Anonymization Tools

Additional Open-Source Tools

BMW Anonymization API - based training/inference solutions that we already published/will publish for Object Detection and Semantic Segmentation |

Infographics

AI Security Market Map

![Market Map - for-ai-genai-risks-and-the-emerging-startup-landscape/)

ML Code Security

HiddenLayer: Model as Code - Research about some vectors in ML libraries
lintML - Security linter for ML, by Nvidia
differential-privacy-library - Library designed for differential privacy and machine learning
HiddenLayer: Model as Code - Research about some vectors in ML libraries

MLOps Infrastructure Vulnerabilities

SILENT SABOTAGE - Study on bot compromise for converting Pickle to SafeTensors
NOT SO CLEAR: HOW MLOPS SOLUTIONS CAN MUDDY THE WATERS OF YOUR SUPPLY CHAIN - Study on vulnerabilities for the ClearML platform
Uncovering Azure's Silent Threats: A Journey into Cloud Vulnerabilities - Study on security issues of Azure MLAAS
The MLOps Security Landscape
Confused Learning: Supply Chain Attacks through Machine Learning Models
SILENT SABOTAGE - Study on bot compromise for converting Pickle to SafeTensors
NOT SO CLEAR: HOW MLOPS SOLUTIONS CAN MUDDY THE WATERS OF YOUR SUPPLY CHAIN - Study on vulnerabilities for the ClearML platform

MlSecOps pipeline

MLSecOps Pipeline

AI Security Study Map

image
image

Open Source Security Tools

Additional Open-Source Tools

TF Encrypted - party computation|
skill-audit-mcp
NB Defense

AI Agent Security

HOL Guard - source local-first runtime security layer for AI agents that evaluates supported tool actions and local artifacts for prompt injection, secret exposure, unsafe commands, package risks, and MCP threats |
Prismor - hosted runtime control plane that screens AI coding-agent tool calls against a signed policy before execution, blocking or routing to human approval secret exfiltration, prompt-injection-driven actions, and destructive commands across Claude Code, Codex, and MCP servers |
Bifrost - native AI gateway that enforces guardrails, rate limits, and fine-grained access controls across LLM and MCP traffic. |
Humanbound - source CLI that sends LLM-generated adversarial attacks to an agent's HTTP endpoint and scores results against OWASP LLM Top 10, OWASP Agentic Top 10, NIST, and EU AI Act mappings |

AI Supply-Chain Security

BomLens - first SBOM generator that builds CycloneDX ML-BOMs for Hugging Face models, with license and known-vulnerability reports |

Embodied AI and Robotics ML Security

BadVLA - language-action models via objective-decoupled optimization; reference implementation for the paper |
Provael - language-action robot policies in simulation; reports attack success rate with a 95% Wilson interval against a benign-control false-positive floor |
roboticAttack - language-action models; code for "Exploring the Adversarial Vulnerabilities of VLA Models in Robotics" |

LLM Security and Red Teaming

Garak

Model Testing, Monitoring, and Evaluation

CircleGuardBench - fledged benchmark for evaluating protection capabilities of AI models|

MLSploit - MLsploit is a cloud framework for interactive experimentation with adversarial machine learning research.
Gandalf Lakera
Offensive ML Playbook
Citadel Lens
Robust Intelligence Continous Validation - Tool for continuous model validation for compliance with standards
ModelScan
TensorFlow Privacy - preserving machine learning algorithms and tools |
Foolbox
Advertorch
Adversarial ML Threat Matrix
CleverHans
AdvBox
Audit AI
Deep Pwning - pwning is a lightweight framework for experimenting with machine learning models with the goal of evaluating their robustness against a motivated adversary|
Privacy Meter - source library to audit data privacy in statistical and machine learning algorithms|