https://github.com/alexdevassy/Machine_Learning_CTF_Challenges

CTF challenges designed and implemented in machine learning applications
https://github.com/alexdevassy/Machine_Learning_CTF_Challenges

adversarial-machine-learning ai aisecurity ctf large-language-models machine-learning machine-learning-security offensive-security penetration-testing vulnerable-llm-application

Last synced: 3 months ago
JSON representation

CTF challenges designed and implemented in machine learning applications

• Host: GitHub
• URL: https://github.com/alexdevassy/Machine_Learning_CTF_Challenges
• Owner: alexdevassy
• Created: 2021-07-05T16:23:53.000Z (over 3 years ago)
• Default Branch: master
• Last Pushed: 2024-08-29T18:08:26.000Z (6 months ago)
• Last Synced: 2024-08-30T19:14:35.883Z (5 months ago)
• Topics: adversarial-machine-learning, ai, aisecurity, ctf, large-language-models, machine-learning, machine-learning-security, offensive-security, penetration-testing, vulnerable-llm-application
• Language: HTML
• Homepage:
• Size: 40.8 MB
• Stars: 92
• Watchers: 2
• Forks: 25
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-sec-challenges - Machine Learning CTF Challenges - A series of CTF challenges to help you understand how to secure AI and ML applications and infrastructure. (Capture The Flag / AI/ML CTFs)

README

        
# Machine Learning CTF Challenges

As the realms of artificial intelligence and machine learning continue to shape our world :earth_asia:, the imperative to assess their influence on cybersecurity intensifies. While many AI/ML breakthroughs in cybersecurity revolve around defense and threat intelligence, such as intelligent :computer: SIEM systems and AI-driven malware detection, an intriguing question arises: "Can researchers harness AI/ML for offensive security, or can they outmaneuver AI/ML algorithms with innovative cybersecurity approaches?" This presents a captivating new horizon in the domain of offensive security.🚀

Within this repository lies an array of engaging CTF (Capture The Flag :triangular_flag_on_post:) challenges meticulously crafted for applications that leverage machine learning algorithms in their backend. The intent of this repository is to emphasize the need for implementation of security measures within machine learning applications, safeguarding 🛡️ them against the ever-evolving threat landscape. It serves as a guiding beacon in fortifying the convergence of technology and security🔒. 

#### CTF Challenges :open_file_folder:

| Name | Category | Description | Difficulty | References

| --- | --- | --- | --- | --- | 

| [Vault](/Vault_ML_CTF_Challenge/) | Web - Model Inversion | Gain access to Vault and fetch Secret (Flag:). | Hard | 
 [OWASP ML03](https://owasp.org/www-project-machine-learning-security-top-10/docs/ML03_2023-Model_Inversion_Attack.html)

| [Dolos](/Dolos_ML_CTF_Challenge/) | Web - Prompt Injection to RCE | Flag is at same directory as of flask app, [FLAG].txt. | Easy | 


 [OWASP LLM01](https://llmtop10.com/llm01/)

[AML.T0051](https://atlas.mitre.org/techniques/AML.T0051/)



| [Dolos II](/DolosII_ML_CTF_Challenge/) | Web - Prompt Injection to SQL Injection | Make the LLM to reveal Secret (Flag:) of user David. | Easy | 


 [OWASP LLM01](https://llmtop10.com/llm01/)

[AML.T0051](https://atlas.mitre.org/techniques/AML.T0051/)



| [Heist](/Heist_ML_CTF_Challenge/) | Web - Data Poisoning Attack | Compromise CityPolice's AI cameras and secure a smooth escape for Heist crew's red getaway car! | Medium | 


[OWASP LLM03](https://llmtop10.com/llm03/)

[OWASP ML02](https://owasp.org/www-project-machine-learning-security-top-10/docs/ML02_2023-Data_Poisoning_Attack.html)

[AML.T0020](https://atlas.mitre.org/techniques/AML.T0020/)



| [Persuade](/Persuade_ML_CTF_Challenge/) | Web - Model Serialization Attack | Flag is at /app/InternalFolder/Flag.txt, not on the website. Find it. | Medium | 


[OWASP LLM05](https://llmtop10.com/llm05/)

[OWASP ML06](https://owasp.org/www-project-machine-learning-security-top-10/docs/ML06_2023-AI_Supply_Chain_Attacks.html)

[AML.T0010](https://atlas.mitre.org/techniques/AML.T0010/)



| [Fourtune](/Fourtune_ML_CTF_Challenge/)  | Web - Model Extraction Attack | Bypass AI Corp's identity verification to view the flag | Hard | 


[OWASP LLM10](https://llmtop10.com/llm10/)

[AML.T0044](https://atlas.mitre.org/techniques/AML.T0044/)



:thought_balloon: If you want to contribute to the above list of CTF challenges (please do), submit a pull request or ping me at [![LinkedIn Badge](https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white)](https://in.linkedin.com/in/alex-devassy-358421138) 

Stay tuned for more challenges being added to the repo. :eyes: