{"id":23934731,"url":"https://github.com/N372unn32/AI-ML-LLM-security-resources","last_synced_at":"2025-09-11T17:32:40.053Z","repository":{"id":226492695,"uuid":"768788316","full_name":"N372unn32/AI-ML-LLM-security-resources","owner":"N372unn32","description":"list of resources for AI/ML/LLM security","archived":false,"fork":false,"pushed_at":"2024-03-17T18:18:02.000Z","size":20,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-03-17T19:30:14.550Z","etag":null,"topics":["ai","aisecurity","llm","ml","mlsecurity","resources","security"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/N372unn32.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-03-07T18:24:53.000Z","updated_at":"2024-03-17T19:30:15.643Z","dependencies_parsed_at":"2024-03-17T19:30:15.453Z","dependency_job_id":"57f85485-26c6-4d8c-aa68-b0658079df47","html_url":"https://github.com/N372unn32/AI-ML-LLM-security-resources","commit_stats":null,"previous_names":["n372unn32/ai-ml-llm-security-resources"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/N372unn32%2FAI-ML-LLM-security-resources","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/N372unn32%2FAI-ML-LLM-security-resources/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/N372unn32%2FAI-ML-LLM-security-resources/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/N372unn32%2FAI-ML-LLM-security-resources/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/N372unn32","download_url":"https://codeload.github.com/N372unn32/AI-ML-LLM-security-resources/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232659164,"owners_count":18557020,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","aisecurity","llm","ml","mlsecurity","resources","security"],"created_at":"2025-01-06T00:35:50.621Z","updated_at":"2025-01-06T00:37:15.165Z","avatar_url":"https://github.com/N372unn32.png","language":null,"funding_links":[],"categories":["资源列表"],"sub_categories":["参考资源"],"readme":"# AI/ML/LLM-security-resources\nbookmarks. list of resources for AI/ML/LLM security\n\n\n## Table of Contents\n\n- [Blogs / PPTs / Sites](https://github.com/N372unn32/AI-ML-security-study-resources/edit/main/README.md#blogs--ppts--sites)\n- [Courses / Videos](https://github.com/N372unn32/AI-ML-security-study-resources/edit/main/README.md#courses--videos)\n- [Books / Papers](https://github.com/N372unn32/AI-ML-security-study-resources/edit/main/README.md#books--papers)\n- [Tools](https://github.com/N372unn32/AI-ML-security-study-resources/edit/main/README.md#tools)\n\n## Blogs / PPTs / Sites\n\n| Title | Author | Link |\n| ----- | ------ | ---- |\n| Blogs at DEFCON AI Village| DEFCON AI Village | [aivillage.org](https://aivillage.org/blog/) |\n| Zen and the Art of Adversarial Machine Learning | Will Pearce, Giorgio Severi  | [blackhat.com](https://i.blackhat.com/EU-21/Thursday/EU-21-Pearce-Zen-And-The-Art-Of-Adversarial-ML.pdf) |\n| AI Red Team: Machine Learning Security Training | Will Pearce, Joseph Lucas, Rich Harang and John Irwin  | [developer.nvidia.com](https://developer.nvidia.com/blog/ai-red-team-machine-learning-security-training/) |\n| NVIDIA AI Red Team: An Introduction | Will Pearce and Joseph Lucas  | [developer.nvidia.com](https://developer.nvidia.com/blog/nvidia-ai-red-team-an-introduction/) |\n| Increasing transparency in AI security | Mihai Maruseac, Sarah Meiklejohn, Mark Lodato, Google Open Source Security Team (GOSST) | [security.googleblog.com](https://security.googleblog.com/2023/10/increasing-transparency-in-ai-security.html) |\n| PIPE - Prompt Injection Primer for Engineers | jthack | [github.com](https://github.com/jthack/PIPE) |\n| AI-Powered Fuzzing: Breaking the Bug Hunting Barrier | Dongge Liu, Jonathan Metzman, Oliver Chang, Google Open Source Security Team  | [security.googleblog.com](https://security.googleblog.com/2023/08/ai-powered-fuzzing-breaking-bug-hunting.html) |\n| Secure AI FrameworkApproach | Google | [services.google.com](https://services.google.com/fh/files/blogs/google_secure_ai_framework_approach.pdf) |\n| Securing the AI Pipeline | DAN BROWNE, MUHAMMAD MUNEER | [mandiant.com](https://www.mandiant.com/resources/blog/securing-ai-pipeline) |\n| Microsoft’s open automation framework to red team generative AI Systems | Ram Shankar Siva Kumar | [microsoft.com](https://www.microsoft.com/en-us/security/blog/2024/02/22/announcing-microsofts-open-automation-framework-to-red-team-generative-ai-systems/) |\n| Microsoft AI Red Team | Microsoft Learn | [learn.microsoft.com](https://learn.microsoft.com/en-us/security/ai-red-team/) |\n| OWASP Machine Learning Security Top Ten | OWASP | [owasp.org](https://owasp.org/www-project-machine-learning-security-top-10/) |\n| OWASP AI Top Ten | OWASP | [owasp.org](https://owasp.org/www-project-ai-top-ten/) |\n| OWASP Top 10 for Large Language Model Applications | OWASP | [owasp.org](https://owasp.org/www-project-top-10-for-large-language-model-applications/) |\n| Adversarial ML Threat Matrix | MITRE | [github.com](https://github.com/mitre/advmlthreatmatrix) |\n| Welcome to the Offensive ML Playbook | @whitehacksec | [wiki.offsecml.com](https://wiki.offsecml.com/Welcome+to+the+Offensive+ML+Playbook) |\n\n\n## Courses / Videos\n\n| Title | Author | Link |\n| ----------- | -------- | ---- |\n| AI Application Security: Understanding Prompt Injection Attacks and Mitigations | rez0 | [youtube.com](https://www.youtube.com/watch?v=MxxPbN9GGYE) |\n| Red Teaming LLMs with Jupyter Notebooks: A Practical Guide | Pete Bryan | [youtube.com (Timestamp - 2:12)](https://www.youtube.com/watch?v=5CK-hpSYOkQ) |\n| Learn from Microsoft’s AI Red Team on how to make your organization safer | Gary Lopez | [brighttalk.com](https://www.brighttalk.com/webcast/10415/607319) |\n\n\n## Books / Papers\n\n| Title | Author | Link |\n| ----------- | -------- | ---- |\n| Jailbreaking Black Box Large Language Models in Twenty Queries | Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, Eric Wong | [arxiv.org](https://arxiv.org/abs/2310.08419) |\n| Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations | Apostol Vassilev, Alina Oprea, Alie Fordyce, Hyrum Anderson| [nvlpubs.nist.gov](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-2e2023.pdf) |\n| Not with a Bug, But with a Sticker: Attacks on Machine Learning Systems and What To Do About Them | Ram Shankar Siva Kumar, Hyrum Anderson| [amazon.com](https://www.amazon.com/Not-Bug-But-Sticker-Learning/dp/1119883989) |\n\n\n## Tools\n\n| Title | Author | Link |\n| ----------- | ------ | ---- |\n| Python Risk Identification Tool for generative AI (PyRIT) | Azure | [github.com](https://github.com/Azure/PyRIT) |\n| Counterfit | Azure | [github.com](https://github.com/Azure/counterfit) |\n| garak, LLM vulnerability scanner | leondz | [github.com](https://github.com/leondz/garak) |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FN372unn32%2FAI-ML-LLM-security-resources","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FN372unn32%2FAI-ML-LLM-security-resources","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FN372unn32%2FAI-ML-LLM-security-resources/lists"}