{"id":13633782,"url":"https://github.com/protectai/ai-exploits","last_synced_at":"2025-05-14T14:07:25.808Z","repository":{"id":207604934,"uuid":"709957829","full_name":"protectai/ai-exploits","owner":"protectai","description":"A collection of real world AI/ML exploits for responsibly disclosed vulnerabilities ","archived":false,"fork":false,"pushed_at":"2024-10-23T20:40:54.000Z","size":83,"stargazers_count":1571,"open_issues_count":3,"forks_count":130,"subscribers_count":37,"default_branch":"main","last_synced_at":"2025-04-12T13:57:33.669Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/protectai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-25T18:10:42.000Z","updated_at":"2025-04-11T21:26:33.000Z","dependencies_parsed_at":"2023-11-21T16:54:36.092Z","dependency_job_id":"a7171b37-67a9-48a3-b88c-1426c8090bf4","html_url":"https://github.com/protectai/ai-exploits","commit_stats":null,"previous_names":["protectai/ai-exploits"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/protectai%2Fai-exploits","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/protectai%2Fai-exploits/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/protectai%2Fai-exploits/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/protectai%2Fai-exploits/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/protectai","download_url":"https://codeload.github.com/protectai/ai-exploits/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254159194,"owners_count":22024558,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T23:00:51.676Z","updated_at":"2025-05-14T14:07:25.789Z","avatar_url":"https://github.com/protectai.png","language":"Python","funding_links":[],"categories":["Open Source Security Tools","Python","LLM SECURITY / AI SECURITY","Table of Contents","Tools of Trade","GPT Security"],"sub_categories":["LLM Vulnerability Testing","🤖 AI Security / AI Red Teaming","Offensive / Red Teaming","Bug Bounty"],"readme":"\u003cdiv align=\"center\"\u003e\n\n# AI Exploits\n\n  \u003cimg width=\"250\" src=\"https://github.com/protectai/ai-exploits/assets/5151193/aef11c4a-d758-45fe-aab8-c9df714cdbe5\" alt=\"AI Exploits Logo\"\u003e\n\n\u003c/div\u003e\n\nThe AI world has a security problem and it's not just in the inputs given to LLMs such as ChatGPT. Based \non research done by [Protect AI](https://protectai.com) and independent security experts on the [Huntr](https://huntr.com) Bug Bounty Platform, there are far more impactful and practical attacks \nagainst the tools, libraries and frameworks used to build, train, and deploy machine learning models. Many of these \nattacks lead to complete system takeovers and/or loss of sensitive data, models, or credentials most often without the need\nfor authentication. \n\nWith the release of this repository, [Protect AI](https://protectai.com) hopes to demystify to the Information Security community what practical attacks against AI/Machine Learning infrastructure look like in the real world and raise awareness to the amount of vulnerable components that currently exist in the AI/ML ecosystem. More vulnerabilities can be found here:\n* [November Vulnerability Report](https://protectai.com/threat-research/november-vulnerability-report)\n* [December Vulnerability Report](https://protectai.com/threat-research/december-vulnerability-report)\n* [January Vulnerability Report](https://protectai.com/threat-research/january-vulnerability-report)\n* [February Vulnerability Report](https://protectai.com/threat-research/february-vulnerability-report)\n* [March Vulnerability Report](https://protectai.com/threat-research/march-vulnerability-report)\n* [April Vulnerability Report](https://protectai.com/threat-research/april-vulnerability-report)\n* [May Vulnerbility Report](https://protectai.com/threat-research/may-vulnerability-report)\n* [June Vulnerbility Report](https://protectai.com/threat-research/june-vulnerability-report)\n* [July Vulnerbility Report](https://protectai.com/threat-research/july-vulnerability-report)\n\n## Overview\n\nThis repository, **ai-exploits**, is a collection of exploits and scanning templates for responsibly disclosed vulnerabilities affecting machine learning tools.\n\nEach vulnerable tool has a number of subfolders containing three types of utilities: [Metasploit](https://github.com/rapid7/metasploit-framework) modules, [Nuclei](https://github.com/projectdiscovery/nuclei) templates\nand CSRF templates. Metasploit modules are for security professionals looking to exploit the vulnerabilities and Nuclei templates are for scanning a large number of remote servers to determine if they're vulnerable.\n\n## Demo\n\nVideo demonstrating running one of the Metasploit modules against Ray:\n\n[![Exploit Demo](https://img.youtube.com/vi/5aSwPQKKhi4/0.jpg)](https://youtu.be/5aSwPQKKhi4)\n\n## Setup \u0026 Usage\n\nThe easiest way to use the modules and scanning templates is to build and run the Docker image provided by the `Dockerfile` in this repository. The Docker image will have Metasploit and Nuclei already installed along with all the necessary configuration.\n\n###  Docker\n\n1. Build the image:\n\n   ```bash\n   git clone https://github.com/protectai/ai-exploits \u0026\u0026 cd ai-exploits\n   docker build -t protectai/ai-exploits .\n   ```\n\n2. Run the docker image:\n   \n   ```bash\n   docker run -it --rm protectai/ai-exploits /bin/bash\n   ```\n\nThe latter command will drop you into a `bash` session in the container with `msfconsole` and `nuclei` ready to go.\n\n### Using the Metasploit Modules\n\n#### With Docker\n\nStart the Metasploit console (the new modules will be available under the `exploits/protectai` category), load a module, set the options, and run the exploit.\n\n   ```bash\n   msfconsole\n   msf6 \u003e use exploit/protectai/ray_job_rce\n   msf6 exploit(protectai/ray_job_rce) \u003e set RHOSTS \u003ctarget IP\u003e\n   msf6 exploit(protectai/ray_job_rce) \u003e run\n   ```\n\n#### With Metasploit Installed Locally\n\nCreate a folder `~/.msf4/modules/exploits/protectai` and copy the exploit modules into it.\n\n   ```bash\n   mkdir -p ~/.msf4/modules/exploits/protectai\n   cp ai-exploits/ray/msfmodules/* ~/.msf4/modules/exploits/protectai\n   msfconsole\n   msf6 \u003e use exploit/protectai/\u003cexploit_name.py\u003e\n   ```\n\n### Using Nuclei Templates\n\nNuclei is a vulnerability scanning engine which can be used to scan large numbers of servers for known vulnerabilities in web applications and networks.\n\nNavigate to nuclei templates folder such as `ai-exploits/mlflow/nuclei-templates`. In the Docker container these are stored in the `/root/nuclei-templates` folder. Then simply point to the template file and the target server.\n   ```\n   cd ai-exploits/mlflow/nuclei-templates\n   nuclei -t mlflow-lfi.yaml -u http://\u003ctarget\u003e:\u003cport\u003e\n   ```\n\n### Using CSRF Templates\n\nCross-Site Request Forgery (CSRF) vulnerabilities enable attackers to stand up a web server hosting a malicious HTML page \nthat will execute a request to the target server on behalf of the victim. This is a common attack vector for exploiting \nvulnerabilities in web applications, including web applications which are only exposed on the localhost interface and \nnot to the broader network. Below is a simple demo example of how to use a CSRF template to exploit a vulnerability in a \nweb application.\n\nStart a web server in the csrf-templates folder. Python allows one to stand up a simple web server in any \ndirectory. Navigate to the template folder and start the server.\n\n   ```bash\n   cd ai-exploits/ray/csrf-templates\n   python3 -m http.server 9999\n   ```\n\nNow visit the web server address you just stood up (http://127.0.0.1:9999) and hit F12 to open \nthe developer tools, then click the Network tab. Click the link to ray-cmd-injection-csrf.html. You should see that \nthe browser sent a request to the vulnerable server on your behalf.\n\n## Contribution Guidelines\n\nWe welcome contributions to this repository. Please read our [Contribution Guidelines](CONTRIBUTING.md) for more information on how to contribute.\n\n## License\n\nThis project is licensed under the [Apache 2.0 License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprotectai%2Fai-exploits","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprotectai%2Fai-exploits","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprotectai%2Fai-exploits/lists"}