{"id":15175458,"url":"https://github.com/cohere-ai/cohere-terrarium","last_synced_at":"2025-04-09T13:05:31.819Z","repository":{"id":235856434,"uuid":"791358094","full_name":"cohere-ai/cohere-terrarium","owner":"cohere-ai","description":"A simple Python sandbox for helpful LLM data agents","archived":false,"fork":false,"pushed_at":"2024-06-18T16:56:00.000Z","size":320,"stargazers_count":242,"open_issues_count":5,"forks_count":37,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-04-02T12:08:24.766Z","etag":null,"topics":["code-interpreter","llm-agent","sandbox"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cohere-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-24T15:17:01.000Z","updated_at":"2025-04-01T14:56:40.000Z","dependencies_parsed_at":"2024-04-24T19:15:46.173Z","dependency_job_id":"cf1fba5e-7102-4b0e-a1b7-05c1dbfd1396","html_url":"https://github.com/cohere-ai/cohere-terrarium","commit_stats":{"total_commits":2,"total_committers":1,"mean_commits":2.0,"dds":0.0,"last_synced_commit":"179d9cba1fcd5c60573bed8391fbefdfb2a4a63c"},"previous_names":["cohere-ai/cohere-terrarium"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cohere-ai%2Fcohere-terrarium","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cohere-ai%2Fcohere-terrarium/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cohere-ai%2Fcohere-terrarium/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cohere-ai%2Fcohere-terrarium/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cohere-ai","download_url":"https://codeload.github.com/cohere-ai/cohere-terrarium/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248045230,"owners_count":21038553,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["code-interpreter","llm-agent","sandbox"],"created_at":"2024-09-27T12:39:12.581Z","updated_at":"2025-04-09T13:05:31.791Z","avatar_url":"https://github.com/cohere-ai.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Terrarium - A Simple Python Sandbox\n\nTerrarium is a relatively low latency, easy to use, and economical Python sandbox - to be used as a docker deployed container, for example in GCP Cloud Run - for executing untrusted user or LLM generated ``python`` code.\n\n- **Terrarium is fast:** 900ms runtime to generate a 200 dpi png with a simple matplotlib barchart - 500 ms for a svg version. (hosted on GCP Cloud Run)\n- **Terrarium is cheap:** We spent less than $30 a month hosting terrarium on GCP during internal annotations (2GB mem + 1vCPU and at least 1 alive instance + autoscale on demand) \n- **Terrarium is fully compartmentalized:** The sandbox gets completely recycled after every invocation. No state whatsoever is carried over between calls. *Cohere does not give any guarantees for the sandbox integrity.*\n- **Terrarium supports native input \u0026 output files:** You can send any number \u0026 type of files as part of the request and we put it them in the python filesystem. After the code execution we gather up all generated files and return them with the response.\n- **Terrarium supports many common packages:** Terrarium runs on [Pyodide](https://pyodide.org/en/stable/index.html), therefore it supports numpy, pandas, matplotlib, sympy, and other standard python packages.\n\n## Using Terrarium\n\nUsing the deployed Cloud Run is super easy - just call it with the `code` to run \u0026 authorization bearer (if so configured) as follows:\n\n```bash\ncurl -X POST --url \u003cname of your deployed gcp cloud run\u003e \\\n-H \"Authorization: bearer $(gcloud auth print-identity-token)\" \\\n-H \"Content-Type: application/json\" \\\n--no-buffer \\\n--data-raw '{\"code\": \"1 + 1\"}'\n```\n\nwhich returns:\n```json\n{\"output_files\":[],\"final_expression\":2,\"success\":true,\"std_out\":\"\",\"std_err\":\"\",\"code_runtime\":16}\n```\n\nThe authentication `gcloud auth print-identity-token` needs to be renewed every hour.\n\nSee `terrarium_client.py` for an easy-to-use python function to call the service - including file input \u0026 output functionality via base64 encoded files.\n\n## Sandbox Design\n\nThe sandbox is composed of multiple layers: \n\n1. Parse, compile, \u0026 execute python code inside a node.js process - via CPython compiled to webassembly, not running natively - with https://pyodide.org/en/stable/index.html. This approach restricts the untrusted code's abilities: \n    - NO access to the filesystem (pyodide provides a compartmentalized memory only guest filesystem)\n    - NO threading \u0026 multiprocessing\n    - NO ability to call a subprocess \n    - NO access to any of our hosts memory\n    - NO access to other call states: we recycle the full pyodide environment (including the virtual file system, global state, loaded libs ... the works) after every call\n    - NO network nor internet access (this is a current design choice and could be changed in the future)\n\n2. Deploy the node.js host into a GCP Cloud Run container, which restricts:\n    - runtime\n    - decouples the node.js host (in case of a breakout) from the rest of our network\n\n---\n\nThe following packages are supported out of the box:\nhttps://pyodide.org/en/stable/usage/packages-in-pyodide.html including, but not limited to:\n\n- numpy\n- pandas\n- sympy\n- beautifulsoup4\n- matplotlib (plt.show() is not supported, but plt.savefig() works like a charm - most of the time)\n- python-sat\n- scikit-learn\n- scipy\n- sqlite3 (not enabled by default, but we could load it as well)\n\n## Development\n\nYou need node.js installed on your system. To install dependencies run:\n\n```bash\nnpm install\nmkdir pyodide_cache\n```\n\nrun the server \u0026 function locally:\n```bash\nnpm run dev\n```\n\nexecute code in the terrarium:\n```bash\ncurl -X POST -H \"Content-Type: application/json\" \\\n--url http://localhost:8080 \\\n--data-raw '{\"code\": \"1 + 1\"}' \\\n--no-buffer\n```\n\nrun a set of test files (all .py files in ``/test``) through the endpoint with: \n```bash\npython terrarium_client.py http://localhost:8080\n```\n\n## Deployment\n\n### Deploy as Docker container\n\nTo run in docker:\n\n**Build:**\n\n```bash\ndocker build -t terrarium .\n```\n\n**Run:**\n```bash\ndocker run -p 8080:8080 terrarium\n```\n\n**Stop:**\n```bash\ndocker ps\n```\nto get the container id and then\n```bash\ndocker stop {container_id}\n```\n\n### Deploy to GCP Cloud Run \n\nAllocating more resources to speed up run time as well as limiting concurrency from Cloud Run:\n\n```bash\ngcloud run deploy \u003cinsert name of your deployment here\u003e \\\n--region=us-central1 \\\n--source . \\\n--concurrency=1 \\\n--min-instances=3 \\\n--max-instances=100 \\\n--cpu=2 \\\n--memory=4Gi \\\n--no-cpu-throttling \\\n--cpu-boost \\\n--timeout=100\n```\n\n### Handling timeouts\nPyodide today runs on the node.js main process, and can block node.js from responding. Pyodide recommends using a Worker if we need to interrupt. However the interface with pyodide would be through message passing, and it doesn't support matplotlib amongst other libraries.\n\nExample code that would trigger a timeout.\n\n```bash\ncurl -m 110 -X POST \u003cinsert name of your deployment here\u003e \\\n-H \"Authorization: bearer $(gcloud auth print-identity-token)\" \\\n-H \"Content-Type: application/json\" \\\n-d '{\n  \"code\": \"import time\\ntime.sleep(200)\"\n}'\n```\n\nCloud Run doesn't support Dockerfile healthcheck. Once the service is deployed for the first time, you need to grab the service.yaml file and add the liveness probe.\n\n`gcloud run services describe \u003cinsert name of your deployment here\u003e --format export \u003e service.yaml`\n\nAdd [livenessProbe](https://cloud.google.com/run/docs/configuring/healthchecks#yaml_3) after the `image` definition \n\n```\nlivenessProbe:\n  failureThreshold: 1\n  httpGet:\n    path: /health\n    port: 8080\n  periodSeconds: 100\n  timeoutSeconds: 1\n```\nRun `gcloud run services replace service.yaml `\n\nThis is only needed once per new Cloud Run service deployed.\n\nDocker itself doesn't support auto-restarts based on HEALTHCHECK (it seems). Process with pid `1` seems protected, and can't be killed. Would need to spin up a separate service like so: https://github.com/willfarrell/docker-autoheal\n\n\n## Limitations\n\n### Ability to install packages\n\n\n\n### Network access\n\n\n\n### Complex operations\n\nFor large \u0026 complex computations we sometimes observe untraceble \"RangeError: Maximum call stack size exceeded\" exceptions in Pyodide.\n\n- This increasingly happens when we set a too high dpi parameter on png saves for matplotlib figures\n- Or highly complex pandas operations\n\nSee also: https://blog.pyodide.org/posts/function-pointer-cast-handling/","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcohere-ai%2Fcohere-terrarium","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcohere-ai%2Fcohere-terrarium","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcohere-ai%2Fcohere-terrarium/lists"}