{"id":13581347,"url":"https://github.com/camerondurham/codecanvas","last_synced_at":"2026-01-02T16:02:54.284Z","repository":{"id":38377929,"uuid":"449648356","full_name":"camerondurham/codecanvas","owner":"camerondurham","description":"Remote code executor server, frontend, and CLI to run untrusted code as a non-root user in a Docker container.","archived":false,"fork":false,"pushed_at":"2024-12-18T19:00:41.000Z","size":7315,"stargazers_count":37,"open_issues_count":10,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-29T06:48:49.970Z","etag":null,"topics":["code-execution","docker","earthfile","executor","flyio","go","hacktoberfest","non-root","playground","remote-execution","rlimit","sandbox"],"latest_commit_sha":null,"homepage":"https://u64.cam/codecanvas","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/camerondurham.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-19T10:36:57.000Z","updated_at":"2025-02-24T08:34:53.000Z","dependencies_parsed_at":"2022-08-09T03:16:01.118Z","dependency_job_id":"68caa661-23a9-441f-8a1b-c47e41d9b0e0","html_url":"https://github.com/camerondurham/codecanvas","commit_stats":null,"previous_names":["camerondurham/codecanvas"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/camerondurham%2Fcodecanvas","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/camerondurham%2Fcodecanvas/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/camerondurham%2Fcodecanvas/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/camerondurham%2Fcodecanvas/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/camerondurham","download_url":"https://codeload.github.com/camerondurham/codecanvas/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247450193,"owners_count":20940872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["code-execution","docker","earthfile","executor","flyio","go","hacktoberfest","non-root","playground","remote-execution","rlimit","sandbox"],"created_at":"2024-08-01T15:02:00.580Z","updated_at":"2026-01-02T16:02:54.264Z","avatar_url":"https://github.com/camerondurham.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# codecanvas\n\n[![codecov](https://codecov.io/gh/camerondurham/runner/branch/main/graph/badge.svg)](https://codecov.io/gh/camerondurham/runner)\n[![Go Version](https://img.shields.io/github/go-mod/go-version/camerondurham/runner)](https://img.shields.io/github/go-mod/go-version/camerondurham/runner)\n\nPreviously named \"runner\"\n\n## Demo\n\n![gif demo showing python3 nodejs and c++ running code](assets/animation_opt.gif)\n\n\u003e NOTE: please **do not** rely on this API. This is a hobby\n\u003e project for learning and despite best effort to make it definitely can still crash\n\u003e resilient, it does not handle lots of malicious input. If you do find bugs, please do [report them directly](mailto:u64.cam@gmail.com) or submit a quick [issue](https://github.com/camerondurham/runner/issues/new)!\n\n## Intro\n\nThe runner project is to create an interface for users to run their code remotely without having\nto have any compiler on their machine. It is a toy project meant for learning how to build a backend in Go\nand experimenting how to build a somewhat multi-tenant system to run other people's code.\n\n### High Level Architecture Diagram\n\n![runner project high level diagram](assets/runner-diagram-details-bg.png)\n\n### Project State\n\nSequence diagram with rough state of the project.\n\n[Mermaid Live Editor](https://mermaid-js.github.io/mermaid-live-editor/edit#pako:eNqNVMty2zAM_BUMT8mM3Q_wIRc3ubYT96gLTcE2awpQQdBtJpN_L6lnrKZJpING5O4CWIJ4No5rNBsT8VdCcvjV26PYpqJt8Ei6vrvboVxQNhDTvvEaoRAqqigwt7A9oTtXBN3TI19RLjb42iqCFPWocGBprFaEVBeJgZAZ2yz6mIgKy9kQYF74kj83t1PEH2IpFp3HXnMMPhMWcq2ViBAsHZM94odwJ1gyHvEgidQ3CK1wG6fMX9E7PqlwCLNN8JP3oJzNGnemAh481T21z2UmL5QOGQj2Yn2w-9Al8hZnQQrszhO0vAXexe1j3v9Bl9QzPQT-PZqxSGKnlmorNexOGEJ2UHCdJeHEfIYbx03rA97O5Gv8MqP_BPku7DDGDWCXUWdwWWCBvScrT2PyUwEDY14tz7D4WjCi5oaLnMTlc_SlaT_FIaa1MCskn33P3h_zN3capIgytP0HMmMtbzCGxlmSF2YJahLKOrmFrvnvnpBgwxeEoeZr4mdOp0vu_cZKdNVaQzn9u2TNt2moJ2rNSVfliyKr6VLlH5b474UaB8hAnze_JW2TXk2Obk7NkboN0HFKYF16oWWK2RKzMg3mCeTrPPGeS8WV0RM2WJnshMkunStT0UvGpbYMrvvaK4vZHGyIuDI2Ke-eyJmNSsIRNIzMAfXyFw9h2JY)\n\n```mermaid\nsequenceDiagram\nClient-\u003e\u003eServer: submits code\n\nloop Check\n      Server-\u003e\u003eServer: validate request format\nend\n\nServer--\u003e\u003eCodeRunner: call CodeRunner.Run()\n\nloop TransformRequest\n      CodeRunner-\u003e\u003eCodeRunner: parse language\n      CodeRunner-\u003e\u003eCodeRunner: create language runtime props\nend\n\nCodeRunner--\u003e\u003eController: submit job to controller\n\nloop FindRunner\n    Controller-\u003e\u003eController: find available runner\n    Controller--\u003e\u003eController: lock runner\n\n\n    loop RunnerExecutionFlow\n        Controller-\u003e\u003eStandard Shell: pre-run hook (compile)\n        Standard Shell--\u003e\u003eController: \n        Controller-\u003e\u003eProcess: execute processor binary\n\n        loop Process\n            Process-\u003e\u003eProcess: set resource limits\n            Process-\u003e\u003eProcess: set non-root uid and gid for user code\n            Process-\u003e\u003eProcess: execute user code\n        end\n\n        Process--\u003e\u003eController: return exit code\n        Controller-\u003e\u003eStandard Shell: remove source code\n        Standard Shell--\u003e\u003eController: return result\n    end\n    Controller--\u003e\u003eController: unlock runner\nend\n\n\n\n\nController--\u003e\u003eCodeRunner: return stdout, stderr, runtime errors\n\nCodeRunner--\u003e\u003eServer: return CodeRunnerOutput\nServer--\u003e\u003eClient: return server transformed response\n\n```\n\n## Development\n\n### Repository Structure\n\nThese components live in the following paths:\n\n- browser front-end: [`web-frontend`](https://github.com/camerondurham/runner/tree/main/web-frontend) (thank you to @arekouzounian for this!)\n- command-line interface: [`cli/runner/`](https://github.com/camerondurham/runner/tree/main/cli/runner) (another thank you to @arekouzounian for this!)\n- API Server: [`api/`](https://github.com/camerondurham/runner/tree/main/api) (thank you to @filipgraniczny for the help!)\n- CodeRunner: [`engine/coderunner`](https://github.com/camerondurham/runner/tree/main/engine/coderunner) (thank you to @siwei-li for the help!)\n- Runner Containers: [`engine/runtime`](https://github.com/camerondurham/runner/tree/main/engine/runtime)\n\n## Dev Environment\n\nEditors:\n\n- [Visual Studio Code](https://code.visualstudio.com/Download)\n- [GoLand](https://www.jetbrains.com/go/) from Jetbrains for free with an [educational license](https://www.jetbrains.com/community/education/#students)\n\nExtensions setup docs:\n\n- Writing Go in VSCode: \u003chttps://code.visualstudio.com/docs/languages/go\u003e\n- Debugging Go in VSCode: \u003chttps://github.com/golang/vscode-go/blob/master/docs/debugging.md\u003e\n\nRecommended extensions (VSCode):\n\nSearch for these extension ids in VSCode and feel free to\nadd your personal favs:\n\n1. `golang.go`\n   for running and debugging Go (see [vscode-go debugging docs](https://github.com/golang/vscode-go/blob/master/docs/debugging.md))\n1. `eamodio.gitlens`\n   git lens (pro tip, enable editor heat map in upper right corner)\n1. `ms-vscode-remote.remote-containers`\n   develop in containers with all dependencies pre-installed\n1. `ms-vscode-remote.remote-wsl`\n   for Windows WSL users\n1. `yzhang.markdown-all-in-one`\n   for writing docs\n\nDocker:\n\nWe will likely end up using Docker and include instructions here. For now, you\ncan install [Docker Desktop](https://www.docker.com/get-started) if you like.\n\n### Using Dev Containers with VSCode (recommended)\n\nTo use a pre-built development container, you can use the VSCode and the dev container provided in `.devcontainer/devcontainer.json`.\nThis approach will use a Docker container with Go, cobra, python3, and g++ pre-installed and ready to use.\n\nHere is a waay to long video with ~5 mins showing setup and total 12 mins\ndemoing using the container: [runner devcontainer setup video](https://youtu.be/f9qBHyzxIlo)\n\nSteps:\n\n1. Verify that you have [Docker](https://www.docker.com/get-started) running\n1. Open VSCode and install the **Remote - Containers** extension: `ms-vscode-remote.remote-containers`\n1. Run the dev container\n   1. Open the Command Palette (cmd + shift + P on macOS, `F1` or ctrl + shift + p on Windows/Linux)\n   1. Run `Remote-Containers: Open Folder in Container`\n   1. Select the `runner` repository folder\n1. Wait for the dev container to start up and open the VSCode Terminal as needed to run commands!\n\nAlso see [Remote-Containers: open an existing folder in a container](https://code.visualstudio.com/docs/remote/containers#_quick-start-open-an-existing-folder-in-a-container).\n\n\n### Development Workflow\n\nThis repository is primarily written in Go and the Makefile has a helper\ncommands to make development easier and more consistent.\n\n\u003e Note: before you start development, please run `make install-hooks`\n\u003e to install [Git Hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks)\n\u003e in your repository's `.git/hooks` directory. This will install a pre-commit\n\u003e hook that automatically formats your code with [gofmt](https://go.dev/blog/gofmt).\n\n### Using the Earthfile\n\n[Earthly](https://earthly.dev/) is a Go CLI that works with Docker. It is build tool that lets you run continuous\nintegration and deployment actions locally. The reason it is being used here is to make CI/CD for this repo easier.\n\nFor this repo in particular, we need some tests to run in a Dockerized Linux environment to\nbe as close to our deployment image as possible. Earthly makes this really easy since it can\ncan run tests as part of its build process.\n\n- [Earthly Website](https://earthly.dev/)\n- [Learn the basics](https://docs.earthly.dev/basics)\n\n#### Installing\n\nFor macOS users:\n```shell\nbrew install earthly/earthly/earthly \u0026\u0026 earthly bootstrap\n```\n\nFor other users: [earthly.dev/get-earthly](https://earthly.dev/get-earthly)\n\n#### Using\n\nTo use, just check the target you want to run in the `Earthfile`. It is effectively like a\nMakefile + Dockerfile and below are a few commands you may want to run during development.\n\n```shell\n# run all tests and lints, just like how they'll be run in CI when you open a PR\nearthly +run-ci\n\n# lint the sourcecode with the golangci lint tool\nearthly +lint\n\n# just test the go code with coverage\nearthly +test-go\n```\n\n### Using the Makefile\n\nBy now, you are probably familiar with Makefiles. If not, this\nwiki provides a great summary: [cs104/wiki/makefile](https://bytes.usc.edu/cs104/wiki/makefile/) (written by Leif Wesche).\n\nHere's a quick summary of what the targets will do:\n\n```bash\n# print out all the makefile targets\nmake\n\n# create or create mocks for unit testing, helpful if you have\n# modified any of the interfaces in a `types.go` file\nmake gen-mocks\n\n# run the API server (blocking, you can't use the terminal anymore)\nmake run-api\n\n# run all tests in the repository\nmake test\n\n# run go fmt on the repository to format your code\nmake fmt\n\n# install git-hooks to automatically format your code before you commit\nmake install-hooks\n```\n\n### CLI Setup\n\nCLI stands for command line interface.\n\n\u003e Note: this step is **not** needed if you are using the [dev container](#dev-container-recommended) since `cobra` is pre-installed in the container.\n\n#### Installing the `cobra` CLI to help with codegen\n\nInstall cobra dependencies: (required to generate new CLI commands)\n\n```bash\ngo install github.com/spf13/cobra/cobra@v1.3.0\n```\n\n#### Adding New Commands\n\nAdd new cobra command\n\n```bash\n# change directories into the CLI sourcecode\ncd cli/runner\n\n# add new subcommand\ncobra add \u003cCHILD_COMMAND\u003e -p \u003cPARENT_COMMAND\u003e\n\n# example:\ncobra add childCommand -p 'parentCommand'\n```\n\n#### Other Resources\n\n- Where the CLI code lives in this repo: [cli/runner](https://github.com/camerondurham/runner/tree/main/cli/runner)\n- [Docs: Cobra Concepts](https://cobra.dev/#concepts)\n- [Docs: Getting Started](https://cobra.dev/#getting-started)\n- Examples:\n  - [adding a command line flag to CLI](https://github.com/camerondurham/ch/blob/4bb750335485169e469bdb191c8ca29bb107b358/cmd/create.go#L171)\n  - [reading what user set flag to](https://github.com/camerondurham/ch/blob/4bb750335485169e469bdb191c8ca29bb107b358/cmd/create.go#L75)\n\n### Running the Server\n\nDuring CLI or even server development, you will likely want to run the server during testing.\n\nIn the root directory `runner`, you can run the API a couple ways:\n\n```bash\n# 1. use the Makefile\nmake run-api\n\n# 2. just use the go command\ngo run api/main.go\n```\n\nUsually you'll want to run the server in the background to you can do other\nthings with your terminal. However, you'd need to kill the process running on port `10100`\nonce you're done. You can use the `api/kill_server.sh` script for this.\n\n```bash\n# 1. run the API in the background\ngo run api/main.go \u0026\n\n# 2. once you are done, use the script to shut down processes on port 10100\n./api/kill_server.sh\n\n```\n\nYou can also use the `api/kill_server.sh` script if you see this error:\n\n\u003e error starting server: listen tcp :10100: bind: address already in use\n\n### Go Tips\n\n#### Working with Go Modules\n\nGo Module:\n\n```bash\n# you usually will not have to run this since we should already have a go.mod and go.sum file\ngo mod init github.com/\u003cname\u003e/\u003crepo-name\u003e\n\n# add new library\ngo get \u003cnew dependency\u003e\n\n# organize modules and dependencies\ngo mod tidy\n\n# remove dependency\ngo mod edit -dropreplace github.com/go-chi/chi\n```\n\n## Testing\n\n### Unit Tests\n\n**What are unit tests and why do we use them?**\n\nUnit testing is used to help us make sure smaller \"units\" of the code\nwork as expected and handle all expected error cases. This project will end up being pretty\nlarge and we want to use unit tests to verify individual components before\npiecing everything together.\n\nIn [runner_test.go](https://github.com/camerondurham/runner/blob/b594c4e023009d06109eb206c2f3e288dddd5e4c/engine/coderunner/runner_test.go#L25-L38),\nwe mock the response of the runtime to isolate what we are testing and produce\nconsist results without actually calling our \"real code\" in the `runtime`\nmodule.\n\nMore about unit tests: [Definition of a Unit Test](https://www.artofunittesting.com/definition-of-a-unit-test).\n\n**How to generate mocks:**\n\nInstall the Go CLI `mockgen` to create mocks from [Go interfaces](https://gobyexample.com/interfaces):\n\n```bash\ngo install github.com/golang/mock/mockgen@v1.6.0\n```\n\nUsing Mockgen to create new mocks for testing:\n\nBasic command structure:\n\n```bash\nmockgen -source ./path/to/file/with/filename.go -destinaion ./path/to/write/mocks/filename.go InterfaceName\n```\n\nExample:\n\nIn `engine/runtime/types.go` there is the interface `Runtime` that we would like to mock for unit tests:\n\n```go\ntype Runtime interface {\n  RunCmd(runprops *RunProps) (*RunOutput, error)\n}\n```\n\nThe command below will create a mock-able `Runtime` interface, helper functions to implement\n`Runtime` that you can call `RunCmd` on.\n\nWe can organize mocks in a submodule by making the `engine/runtime/mocks` directory and provide that and a filename to write the mocked classes.\n\n```bash\nmockgen -source ./engine/runtime/types.go -package=mocks -destination ./engine/runtime/mocks/Runtime.go Runtime\n```\n\nYou can see an example [here](https://github.com/camerondurham/runner/blob/b594c4e023009d06109eb206c2f3e288dddd5e4c/engine/coderunner/runner_test.go#L25-L38)\nof how to actually use mocks in a unit test.\n\n\u003e Note: The command above has been added to the `Makefile`. If you are creating mocks\n\u003e you want for a new file or interface, feel free to add those commands to the\n\u003e `gen-mocks` target so these are generated when you run `make gen-mocks`.\n\n## Integration Testing\n\nWill add more about this later! Here's some [reading](https://martinfowler.com/bliki/IntegrationTest.html) from Martin Fowler for now!\n\nFor now the only sort of \"end-to-end\" integration test is here: [runner/blob/main/engine/integ_test/integration_test.go](https://github.com/camerondurham/runner/blob/main/engine/integ_test/integration_test.go)\n\n## Documentation\n\nWhen writing instructions for users and in the README, please follow syntax recommended by [google developer docs](https://developers.google.com/style/code-syntax)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcamerondurham%2Fcodecanvas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcamerondurham%2Fcodecanvas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcamerondurham%2Fcodecanvas/lists"}