{"id":26138390,"url":"https://github.com/stacklok/codegate-demonstration","last_synced_at":"2025-06-23T03:44:26.896Z","repository":{"id":272632211,"uuid":"892271772","full_name":"stacklok/codegate-demonstration","owner":"stacklok","description":"An intentionally insecure repository of code and secrets used to demonstrate codegate being awesome!","archived":false,"fork":false,"pushed_at":"2025-02-19T13:40:58.000Z","size":3166,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-11T01:57:59.851Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://docs.codegate.ai/quickstart","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stacklok.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-21T20:04:02.000Z","updated_at":"2025-02-10T16:32:43.000Z","dependencies_parsed_at":"2025-01-15T19:13:54.037Z","dependency_job_id":"4c6e0939-a482-443a-b3c9-1ddbbf021607","html_url":"https://github.com/stacklok/codegate-demonstration","commit_stats":null,"previous_names":["stacklok/codegate-demonstration"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/stacklok/codegate-demonstration","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stacklok%2Fcodegate-demonstration","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stacklok%2Fcodegate-demonstration/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stacklok%2Fcodegate-demonstration/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stacklok%2Fcodegate-demonstration/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stacklok","download_url":"https://codeload.github.com/stacklok/codegate-demonstration/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stacklok%2Fcodegate-demonstration/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261407207,"owners_count":23153916,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-11T01:58:01.848Z","updated_at":"2025-06-23T03:44:21.882Z","avatar_url":"https://github.com/stacklok.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ⚠️ Really Insecure Demo Application ⚠️\n\n![A group of people in the woods, reacting in horror to some unseen threat](./images/horror.png)\n\nThis is a deliberately insecure demonstration project used to showcase various\nsecurity vulnerabilities and bad practices thwarted by the CodeGate project.\n\nIt is designed purely for demo purposes to help illustrate common security\npitfalls that LLM code generation workflows are prone to.\n\n## 🚫 CRITICAL SECURITY WARNING\n\n**DO NOT:**\n\n- Install this application in any production environment\n- Use any of this code in real applications\n- Install the `invokehttp` package referenced in this project (it's actually no\n  longer on PyPi, but for all we know you may be running a mirror, so please\n  don't!)\n- Run this code on any public-facing servers\n- Use any of the security practices demonstrated here in your own projects\n\nSeriously, there are some bad things in here! But you're safe, as long as you\nfollow what we do!\n\n## CodeGate to the rescue! 🦸‍♂️\n\nOnce you have CodeGate installed, you can start to explore some of the security\nvulnerabilities present in this code repository and how even the most famed of\nAI large language models (LLMs) will totally miss them!\n\nIn fact, worse still you will see how LLMs can be used to generate code that\nintroduces these vulnerabilities.\n\nYou will also see how generative AI tooling will send secrets stored on your\nmachine directly to the cloud of the provider of the inference service (GitHub\nCopilot, Anthropic, OpenAI, etc). This is a serious security risk and should be\navoided at all costs, yet people fall prey to this every day when using these\ntools.\n\n## Continue extension\n\nYou will need to install the Continue extension in your VS Code editor. You can\ndo this by searching for \"Continue\" in the Extensions Marketplace or by using\nthe installation script provided in the\n[codegate repository](https://github.com/stacklok/codegate).\n\n## Security exfiltration\n\nMany CodeGen AI extensions unwittingly exfiltrate secrets from your machine to\nthe cloud of the provider of the inference service. They do this because the\ntools benefit from gathering as much context as they can on the code they are to\ngenerate from. This typically includes the entire codebase of the project.\n\nCodeGate will protect you from this by ensuring that no secrets leave your\ncontrol, by blocking the LLM prompt from ever leaving your system.\n\n### Demonstration\n\nWithin the Continue chat window, you can type the '@File' command to load a\nlocal file for processing, in this case the `conf.ini` file. This file contains\na few secrets that we want to keep secret. Don't worry about these being exposed\nas they are just mock secrets for demonstration purposes.\n\n![Screenshot of the @File context command being used in the Continue plugin for VS Code](images/file.png)\n\nLoad the file and hit enter!\n\nYou will immediately see that the secrets are blocked and not sent to the LLM\ninference service.\n\n![Screenshot of the Continue plugin with a CodeGate response](images/secrets-blocked.png)\n\n### I'm feeling adventurous!\n\nIf you have Cursor installed, open this repository, try to use the same method\nand see what happens.\n\n![Screenshot of Cursor's response to the secrets file](./images/cursor.png)\n\nHmm, it informed you that the secrets were present, but unfortunately it did not\ndo much to stop them from being sent to their cloud server. This is a serious\nsecurity risk and should be avoided at all costs. You should always prevent\nkeys, tokens, and other secrets from being leaked.\n\n### One more? Sure!\n\nHow about GitHub Copilot (taps fingers Mr. Burns style), a hugley popular tool\nthat is used by many developers. Surely that won't let secrets through?\n\n![Screenshot of Copilot's response to the secrets file](./images/copilot.png)\n\nHuh? What happened? Well nothing happened, the secrets were sent to the GitHub\ncloud, and Copilot was not much help from there, at least Cursor gave us a\nwarning! But zlich, nada, nothing from Copilot at all?!\n\nSo to wrap up, in both cases, the secrets are let through and sent to the cloud\nbased inference service (aka, someone else's computer).\n\n### Malicious packages\n\nUp until now the local LLM folks have been smiling and nodding, but now we are\ngoing to delve into an area that is also a risk to them, malicious packages!\n\nMalicious packages are a real threat to developers. They can be used to\ncompromise your system and steal your data.\n\nThe `invokehttp` package is a malicious Python package that can be used to\ncompromise your system. It is used in this project to demonstrate how easy it is\nto introduce malicious packages into your project.\n\nThis particular attack was used by North Korean hackers to compromise developers\nlooking for a new job. Mock interviews were set up with developers via LinkedIn\nand the developers were asked to install the `invokehttp` package within a\nrepository made to look like a coding challenge. This package was then used to\ncompromise the developer's machine.\n\n#### CodeGate to the rescue!\n\nLet's see what happens when we reference code that uses the `invokehttp` package\non an IDE extension that uses CodeGate.\n\nPerform the same '@File' command as before to load the `python/app.py` file and\nhit enter (don't worry, this won't execute the code, it will just load it into\nthe IDE extension and send it to an LLM inference service).\n\n![Screenshot of a CodeGate security analysis response](./images/invokehttp-codegate.png)\n\nOK, so a bit to unpack here. The code is sent to the LLM inference service,\nhowever the `invokehttp` package is not installed on the system, so the code\nwill not execute. This is a good thing, as the `invokehttp` package is a\nmalicious package that can cause nasty things to happen. In fact the package\ndoes not exist in the registry, so it is not even possible to install it (PyPI\nremoved it after Stacklok reported it).\n\nWhat you will notice though is all of the useful information that is provided.\nFirst is a link to [Stacklok Insight](https://insight.stacklok.com), a free\nservice that provides information about the security of packages. The second is\na warning that the package is malicious and should not be installed.\n\nThis is a great example of how CodeGate can be used to protect developers from\nmalicious packages. This also extends to other suspicious packages or those that\nare no longer maintained.\n\nYou will also see that CodeGate recommended alternative packages that can be\nused in place of the malicious package, along with some helpful code snippets to\nget you started.\n\nLast of all it references materials such as the OWASP Dependency Check system,\nanother great source of information alongside Stacklok Insight.\n\n#### Let's try this with Copilot\n\nSurely GitHub Copilot will be able to help us out here, right? Let's see what\nhappens when we pass it the same code.\n\nFirst off we don't get much back?\n\n![Copilot response to the packages file, first attempt](./images/copilot1.png)\n\nHuh, I literally gave it the code. Why did it not give me anything back?\n\nLet's try some more...\n\n![Copilot response to the packages file, second attempt with more direction](./images/copilot2.png)\n\nOh dear, it seems that GitHub Copilot is not able to help us out here. I will\npaste the code in and try to help it out.\n\n![Copilot response to the packages file, third attempt](./images/copilot3.png)\n\nFinally we got something, but it is not very helpful. It is not able to detect\nthe malicious package and does not provide any information about it. It does not\nprovide any information on the security of the package, nor does it provide any\nrecommendations on alternative packages.\n\nYou now get a feel of how CodeGate has your back, you focus on code and let\nCodeGate focus on security!\n\n## What next?\n\nIf you want to play more, you can reference some of the insecure code examples\nin the rest of this repository. You can also try to introduce some of your own\ninsecure code examples and see how CodeGate can help you out.\n\nIf you want to learn more about CodeGate, you can go to\n[codegate.ai](https://codegate.ai) or come over to our\n[Discord](https://discord.gg/stacklok) and chat with us.\n\n## ⚖️ Legal Disclaimer\n\nThis software is provided for educational purposes only. Using this code in\nproduction environments or using it to attack systems you don't own is strictly\nprohibited. The authors take no responsibility for misuse of this software.\n\n\u003c!-- markdownlint-disable-file MD026 --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstacklok%2Fcodegate-demonstration","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstacklok%2Fcodegate-demonstration","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstacklok%2Fcodegate-demonstration/lists"}