{"id":14968102,"url":"https://github.com/defelo/sandkasten","last_synced_at":"2025-04-06T07:09:59.528Z","repository":{"id":160947585,"uuid":"624570083","full_name":"Defelo/sandkasten","owner":"Defelo","description":"Run untrusted code in an isolated environment","archived":false,"fork":false,"pushed_at":"2025-04-05T15:43:21.000Z","size":1713,"stargazers_count":37,"open_issues_count":2,"forks_count":4,"subscribers_count":2,"default_branch":"develop","last_synced_at":"2025-04-05T16:32:04.223Z","etag":null,"topics":["code-execution","hacktoberfest","isolation","nix","nixos","nsjail","openapi","rest-api","rust","sandbox"],"latest_commit_sha":null,"homepage":"https://sandkasten.bootstrap.academy","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Defelo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-06T19:08:17.000Z","updated_at":"2025-04-05T15:43:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"389ec3a1-4520-43e9-a0c0-ad5f979fbcdb","html_url":"https://github.com/Defelo/sandkasten","commit_stats":{"total_commits":695,"total_committers":5,"mean_commits":139.0,"dds":0.5064748201438849,"last_synced_commit":"f66c21a72fbcc41226423647625c21c63dbe3f4f"},"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Defelo%2Fsandkasten","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Defelo%2Fsandkasten/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Defelo%2Fsandkasten/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Defelo%2Fsandkasten/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Defelo","download_url":"https://codeload.github.com/Defelo/sandkasten/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247445669,"owners_count":20939958,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["code-execution","hacktoberfest","isolation","nix","nixos","nsjail","openapi","rest-api","rust","sandbox"],"created_at":"2024-09-24T13:39:17.837Z","updated_at":"2025-04-06T07:09:59.505Z","avatar_url":"https://github.com/Defelo.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![check](https://github.com/Defelo/sandkasten/actions/workflows/check.yml/badge.svg)](https://github.com/Defelo/sandkasten/actions/workflows/check.yml)\n[![test](https://github.com/Defelo/sandkasten/actions/workflows/test.yml/badge.svg)](https://github.com/Defelo/sandkasten/actions/workflows/test.yml)\n[![codecov](https://codecov.io/gh/Defelo/sandkasten/branch/develop/graph/badge.svg?token=Y5CE2887KO)](https://codecov.io/gh/Defelo/sandkasten)\n![Version](https://img.shields.io/github/v/tag/Defelo/sandkasten?include_prereleases\u0026label=version)\n[![dependency status](https://deps.rs/repo/github/Defelo/sandkasten/status.svg)](https://deps.rs/repo/github/Defelo/sandkasten)\n\n# Sandkasten\nRun untrusted code in an isolated environment\n\n## What is this?\nSandkasten is a code execution engine for running arbitrary untrusted/harmful code in a sandbox,\nisolating it from both the host system and other Sandkasten jobs. A simple REST API allows uploading\nand executing arbitrary programs, while also enabling the user to specify resource limits and\nproviding feedback on the actual resources used. This project was partly inspired by\n[Piston](https://github.com/engineer-man/piston) and aims to solve some problems with it.\n\n## How does it work?\nSandkasten uses [nsjail](https://github.com/google/nsjail) to run programs in restricted\nenvironments and to enforce the specified resource limits. Additionally\n[GNU Time](https://www.gnu.org/software/time/) is used for reporting the resources used by the\nprogram. Programs are always run in a chroot environment using nsjail, which contains only the\nfollowing directories:\n\n- `/program` (rw in compile steps, ro in run steps) contains the compiled program\n- `/box` (ro) current working directory which contains the specified files for compile/run steps\n- `/tmp` (rw, tmpfs)\n- the paths in `/nix/store` that are needed by the selected environment (ro mount from host)\n- some files in `/dev` and `/etc` which are needed for some packages to work properly\n\nPrograms are uniquely identified using the hash value of their source files and selected\nenvironments. If a program has been uploaded and compiled before and is then uploaded again, the\nsame program id is used and the existing compilation results can be used without having to recompile\nthe program.\n\n## Features\n- [x] Compile and execute arbitrary programs.\n- [x] Cache compilation results to avoid having to recompile the same programs for every time they\n      are run.\n- [x] Set resource limits for both compile and run steps.\n- [x] Report resource usage for both compile and run steps.\n- [x] Packages are defined using [Nix](https://nixos.org/).\n- [x] Programs are deleted automatically if they are not executed anymore.\n- [x] Specify stdin, command line arguments and files in the working directory for run steps.\n- [x] Specify environment variables for both compile and run steps.\n- [x] Client library for Rust ([crate](https://crates.io/crates/sandkasten-client), [documentation](https://docs.rs/sandkasten-client))\n- [x] Optional [Prometheus](https://prometheus.io/docs/introduction/overview/) metrics on `/metrics`\n\n### Planned/Ideas\n- [ ] Communicate with running programs via websockets.\n- [ ] Spawn multiple processes that can communicate with each other.\n- [ ] JWTs for individual limits (+rate limits).\n- [ ] Add more packages.\n\n## API Documentation\nOn a running Sandkasten instance, the API documentation is available on `\u003cinstance\u003e/docs` and\n`\u003cinstance\u003e/redoc`. There is also an OpenAPI specification available on `\u003cinstance\u003e/openapi.json`.\n\n## Public Instance\nA public test instance is available at https://sandkasten.bootstrap.academy/. Please note that there\nis a rate limit of 20 requests per minute (if you exceed this limit, you may receive 429 errors).\nAlso, this instance is not intended for production use, it is currently recommended to host your own\ninstance for that (see instructions below).\n\n## Setup instructions\nThe recommended way of installing Sandkasten is to setup a dedicated virtual machine running NixOS.\nTo make this setup easier, this repository contains a basic NixOS configuration template and an\ninstallation script.\n\n### NixOS VM Setup\nThe following steps have been tested on Proxmox VE 7.4-3 x86_64, Proxmox VE 8.0.3 x86_64 and\nlibvirtd 9.4.0 x86_64.\n\n1. Download the minimal NixOS ISO image from https://nixos.org/download.html#nixos-iso\n2. Create a new virtual machine.\n    - Disk size: at least 16GB\n    - Internet connection and DHCP server to get an IPv4 address are required.\n3. Start the virtual machine and boot into the NixOS installer.\n4. Run `sudo su` to obtain root privileges.\n5. If necessary, change the keyboard layout (e.g. `loadkeys de` for german qwertz layout).\n6. Use `lsblk` or `fdisk -l` to find the name of your hard disk.\n7. Run the following commands to download the installation script from GitHub and start the\n    installation. Replace `[disk]` with the path to your hard disk (e.g. `/dev/sda`). Note that\n    this will erase all data on the disk you specify!\n    ```bash\n    curl -o install.sh https://raw.githubusercontent.com/Defelo/sandkasten/latest/install-vm.sh\n    bash install.sh [disk]\n    ```\n    Alternatively you can run the following commands to install the development version.\n    ```bash\n    curl -o install.sh https://raw.githubusercontent.com/Defelo/sandkasten/develop/install-vm.sh\n    FLAKE=github:Defelo/sandkasten/develop#vm bash install.sh [disk]\n    ```\n8. After the script is done, the vm will reboot into the new NixOS installation. The initial root\n    password is `sandkasten` if you want to login via ssh. The Sandkasten server is started\n    automatically and should be listening on `0.0.0.0:80` by default.\n\nIn `/root/sandkasten` you can find a [flake](https://nixos.wiki/wiki/Flakes) which contains the\nconfiguration of your vm. To update Sandkasten and the system packages run\n`nix flake update \u0026\u0026 nixos-rebuild switch --flake .` in this directory. Here you will also find\na `sandkasten.nix` which contains the Sandkasten service configuration, a `configuration.nix`\nwhich contains the system configuration and a `hardware-configuration.nix` which is generated\nautomatically by the install script. After making changes to any of these files run\n`nixos-rebuild switch --flake .` to apply them.\n\n### NixOS Module\nFollow these steps if you want to install Sandkasten on an existing (flakes based) NixOS\ninstallation:\n\n1. Add this repository to your flake inputs:\n    ```nix\n    {\n      inputs.sandkasten.url = \"github:Defelo/sandkasten/latest\";\n    }\n    ```\n2. Add the module to your NixOS configuration:\n    ```nix\n    {\n      imports = [sandkasten.nixosModules.sandkasten];\n    }\n    ```\n3. Configure the module:\n    ```nix\n    {\n      services.sandkasten = {\n        enable = true;\n        environments = p: with p; [\n          rust python typescript  # use `all` to install all environments\n        ];\n\n        # example config (for a full list of configuration options, see `config.toml`)\n        settings = {\n          host = \"0.0.0.0\";\n          port = 8080;\n          max_concurrent_jobs = 16;\n          run_limits.time = 10;\n        };\n      };\n    }\n    ```\n\n## Development\n\n### Setup instructions\n\n#### Required software\nThe following components are needed for a working development environment:\n\n- [Rust](https://www.rust-lang.org/) (stable) toolchain\n- [Nix](https://nixos.org/) with [flakes](https://nixos.wiki/wiki/Flakes) enabled\n- [direnv](https://github.com/direnv/direnv) (optional, but recommended)\n\n#### Enter the development shell\nIf you have direnv installed, you can just use\n`direnv allow` to setup your shell for development. Otherwise you can also use `nix develop`\nto enter a development shell. This will add some tools to your `PATH` and set a few environment\nvariables that are needed by Sandkasten and some of the integration tests.\n\n#### Setup nsjail\nThe first time you enter\nthe development shell, you should run the `setup-nsjail` command, which will copy the `nsjail`\nbinary into your current working directoy, `chown` it to `root` and set the `setuid` bit to allow\nSandkasten to run this binary as root without having to run Sandkasten itself as root (but of\ncourse you could also do that).\n\n#### Install Sandkasten packages\nBefore starting Sandkasten, you should setup a Nix profile with the environments that you want to be\navailable on your instance. A full list of installable environments is available at\n[nix/packages](https://github.com/Defelo/sandkasten/tree/develop/nix/packages). To install a\npackage, you can use the following command:\n\n```bash\nnix profile install --profile pkgs .#packages.\u003cpackage-name\u003e\n```\n\nIf you want to install all packages, use `all` for `\u003cpackage-name\u003e`. You can also add, upgrade or\nremove packages later, but you need to restart Sandkasten after doing so. See\n[`nix profile --help`](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-profile.html)\nfor details.\n\n#### Start the application\nIn the development shell you can just use `cargo run` to start Sandkasten.\n\n### Unit tests\nTo run the unit tests, you can just use `cargo test`. This only requires you to have a working rust\ntoolchain, but you should not need to setup nix for this.\n\n### Integration tests\nTo run the integration tests, you can use `cargo test -F nix -- --ignored`. For this to work you\nneed to have a Sandkasten instance running on `127.0.0.1:8000`. You can also specify a different\ninstance via the `TARGET_HOST` environment variable. If you only want to run the integration tests\nthat do not require a nix development shell, you can omit the `-F nix`. In the development shell you\ncan also run the `integration-tests` command to automatically start a temporary sandkasten instance\nand run the integration tests against it. There is also a `cov` command that runs the integration\ntests and writes an html coverage report to `lcov_html/index.html`.\n\n### Packages\nAll packages are defined using nix expressions in\n[nix/packages](https://github.com/Defelo/sandkasten/tree/develop/nix/packages). Each package has a\nunique id, a human-readable name, a version, optionally a script to compile a program, a script to\nrun a program and a test program that is executed as part of the integration tests to ensure that\nthe package is working.\n\n#### Compile scripts\nThe compile script of a package is executed whenever a new program has been uploaded. When this\nscript is run, the current working directory (`/box`) contains all the source files and the command\nline arguments contain the names of the source files in the same order as they were specified by\nthe client (starting with `main_file` which represents the entrypoint into the program). The purpose\nof the compile script is to compile the provided program and store the result (plus any files that\nmay be needed to run the program) in `/program`.\n\nIf a package does not have a compile script, the source files are instead copied directly into the\nprogram directory.\n\n#### Run scripts\nThe run script of a package is executed whenever a program is executed. When this script is run, the\ncurrent working directory (`/box`) contains the files that have been specified in the run step (if\nany) and `/program` contains the files that have been produced by the corresponding compile script\npreviously (or the source files if the packages does not have a compile script). The first command\nline argument is always the name of the `main_file` (which represents the entrypoint into the\nprogram). In most cases, this is only relevant for interpreted languages (like Python) and can be\nignored for most compiled languages. All other command line arguments are the ones specified by the\nclient and should be forwarded to the actual program.\n\n#### Test program\nEvery package should provide a test program that checks the following:\n\n- Multiple source files are working (e.g. `first_file.py` can import `second_file.py`)\n- Reading from stdin is working. The program should assert that the string `stdin` is read from\n  stdin.\n- Command line arguments are working. The program should assert that the only three command line\n  arguments are `foo`, `bar` and `baz`.\n- File system is working. The program should assert that the file `test.txt` in the current working\n  directory contains the string `hello world`.\n\nIf any of these checks fails, the program should exit with a non-zero exit code. Otherwise, if all\nchecks passed, it should exit with exit code zero and print `OK` to stdout.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdefelo%2Fsandkasten","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdefelo%2Fsandkasten","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdefelo%2Fsandkasten/lists"}