{"id":50705400,"url":"https://github.com/cuolm/pi-sbx-llamacpp","last_synced_at":"2026-06-09T11:01:48.953Z","repository":{"id":363525050,"uuid":"1263732150","full_name":"cuolm/pi-sbx-llamacpp","owner":"cuolm","description":"Run Pi coding agent isolated in a Docker Sandbox microVM with a local llama-server as the inference backend","archived":false,"fork":false,"pushed_at":"2026-06-09T08:12:18.000Z","size":5,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-09T10:09:01.728Z","etag":null,"topics":["ai-agent","docker-sandbox","gguf","llama-cpp","llama-server","local-llm","localllama","microvm","pi-agent","pi-coding-agent","sbx"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cuolm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-09T08:07:47.000Z","updated_at":"2026-06-09T08:17:37.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/cuolm/pi-sbx-llamacpp","commit_stats":null,"previous_names":["cuolm/pi-sbx-llamacpp"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/cuolm/pi-sbx-llamacpp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuolm%2Fpi-sbx-llamacpp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuolm%2Fpi-sbx-llamacpp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuolm%2Fpi-sbx-llamacpp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuolm%2Fpi-sbx-llamacpp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cuolm","download_url":"https://codeload.github.com/cuolm/pi-sbx-llamacpp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuolm%2Fpi-sbx-llamacpp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34103357,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-09T02:00:06.510Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agent","docker-sandbox","gguf","llama-cpp","llama-server","local-llm","localllama","microvm","pi-agent","pi-coding-agent","sbx"],"created_at":"2026-06-09T11:01:48.723Z","updated_at":"2026-06-09T11:01:48.938Z","avatar_url":"https://github.com/cuolm.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pi Coding Agent + Docker Sandbox + llama-server: Setup Guide\n\nThis guide runs the Pi coding agent inside an isolated Docker Sandbox (sbx)\nmicroVM, with inference served by a local llama-server on the host machine.\nThe microVM provides hypervisor-level isolation: Pi cannot access host files\noutside the mounted workspace, cannot reach the host keychain or SSH keys, and\ncannot make network requests beyond what is explicitly permitted. The model\nruns on the host GPU at full speed and is not exposed to the network.\n\nOfficial documentation links:\n- sbx: https://docs.docker.com/ai/sandboxes/get-started/\n- Pi coding agent: https://pi.dev / https://github.com/earendil-works/pi\n- llama.cpp: https://github.com/ggml-org/llama.cpp\n\n---\n\n## Architecture\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│  HOST (macOS or Linux)                                      │\n│                                                             │\n│  llama-server on 127.0.0.1:8001                             │\n│                                                             │\n│  sbx proxy on host, port 3128                               │\n│  • enforces allowedDomains rules                            │\n│  • rewrites host.docker.internal to localhost               │\n│    before checking rules                                    │\n│  • forwards allowed requests to host's 127.0.0.1:8001       │\n│                                                             │\n│  ┌─────────────────────────────────────────────────────┐    │\n│  │  microVM (Linux, isolated)                          │    │\n│  │  Pi agent runs here                                 │    │\n│  │  http_proxy = gateway.docker.internal:3128          │    │\n│  │  no_proxy   = localhost,127.0.0.1,::1,...           │    │\n│  │  Calls http://host.docker.internal:8001/v1          │    │\n│  └─────────────────────────────────────────────────────┘    │\n└─────────────────────────────────────────────────────────────┘\n```\n\n**How Pi reaches llama-server across the VM boundary:**\n\nInside the microVM, `localhost` refers to the VM's own loopback. It is also\nlisted in the VM's `no_proxy` variable, so any request to `localhost` skips\nthe proxy entirely and stays inside the VM. `host.docker.internal` resolves\nto the host via the sbx gateway and is not in `no_proxy`, so requests to it\ntravel through the sbx proxy. The proxy rewrites `host.docker.internal` to\n`localhost` — the host's own `127.0.0.1` — and forwards the request to\nllama-server.\n\nThis is why `models.json` uses `host.docker.internal` as the `baseUrl`, and\nwhy `allowedDomains` must list both `host.docker.internal:8001` (what Pi\nsends) and `localhost:8001` (what the proxy checks against after rewriting).\n\n---\n\n## Step 1 — Install sbx\n\nsbx is Docker's standalone CLI for running AI agents in isolated microVMs. It\ndoes not require Docker Desktop.\n\n**macOS**\n\n```bash\nbrew install docker/tap/sbx\nsbx login\n```\n\n**Linux (Ubuntu/Debian)**\n\n```bash\ncurl -fsSL https://get.docker.com | sudo REPO_ONLY=1 sh\nsudo apt-get install docker-sbx\nsudo usermod -aG kvm $USER\nnewgrp kvm\nsbx login\n```\n\nThe `kvm` group membership is required on Linux because sbx uses KVM for\nhardware-accelerated microVM isolation.\n\nOn first login, sbx prompts you to choose a default network policy. Select\n**Balanced** — it allows common development traffic while blocking everything\nelse by default.\n\nFor other Linux distributions or manual binary installation, see:\nhttps://docs.docker.com/ai/sandboxes/get-started/\n\n---\n\n## Step 2 — Install llama.cpp\n\n**macOS**\n\n```bash\nbrew install llama.cpp\n```\n\n**Linux**\n\nHomebrew also works on Linux:\n\n```bash\nbrew install llama.cpp\n```\n\nAlternatively, build from source:\n\n```bash\ngit clone https://github.com/ggml-org/llama.cpp\ncd llama.cpp\ncmake -B build\ncmake --build build --config Release\nsudo cmake --install build\n```\n\nOn Linux with an Nvidia GPU, add `-DGGML_CUDA=ON` to the first cmake command\nto enable CUDA acceleration. For AMD GPUs use `-DGGML_ROCM=ON`.\n\nFor all installation options see: https://github.com/ggml-org/llama.cpp\n\n---\n\n## Step 3 — Start llama-server on the host\n\nllama-server exposes an OpenAI-compatible HTTP API. On macOS it uses Metal\nfor GPU acceleration. On Linux it uses CUDA or ROCm depending on your GPU.\nThe microVM has no access to the host GPU, which is why inference runs on the\nhost.\n\n```bash\nllama-server \\\n    --hf-repo unsloth/gemma-4-12B-it-qat-GGUF:UD-Q4_K_XL \\\n    --no-mmproj \\\n    --reasoning on \\\n    --temp 1.0 \\\n    --top-p 0.95 \\\n    --top-k 64 \\\n    --alias \"unsloth/gemma-4-12B-it-qat-GGUF\" \\\n    --port 8001\n```\n\n---\n\n## Step 4 — Create the kit directory\n\nAn sbx kit is a directory containing a `spec.yaml` and an optional `files/`\nsubtree. Files placed under `files/home/` are injected into the agent's home\ndirectory inside the microVM at sandbox creation time, before any install\ncommands run.\n\nA kit can be placed anywhere on disk and referenced via the `--kit` flag when\nrunning sbx. This guide places it at `~/.config/docker-sbx/pi-llamacpp`.\n\n```bash\nmkdir -p ~/.config/docker-sbx/pi-llamacpp/files/home/.pi/agent\n```\n\nFull directory layout:\n\n```\n~/.config/docker-sbx/pi-llamacpp/\n├── spec.yaml\n└── files/\n    └── home/\n        └── .pi/\n            └── agent/\n                ├── models.json\n                └── settings.json\n```\n\n---\n\n## Step 5 — Create spec.yaml\n\n```yaml\nschemaVersion: \"1\"\nkind: agent\nname: pi\n\nagent:\n  image: \"docker/sandbox-templates:shell\"\n  entrypoint:\n    run: [pi]\n\nnetwork:\n  allowedDomains:\n    - \"registry.npmjs.org\"\n    - \"host.docker.internal:8001\"\n    - \"localhost:8001\"\n\ncommands:\n  install:\n    - command: \"npm install -g --ignore-scripts @earendil-works/pi-coding-agent\"\n```\n\nBoth `host.docker.internal:8001` and `localhost:8001` are required in\n`allowedDomains` for the reason explained in the Architecture section.\n`registry.npmjs.org` is needed for the npm install during sandbox creation.\n\n---\n\n## Step 6 — Create models.json\n\nPi reads `~/.pi/agent/models.json` inside the microVM to discover custom model\nproviders. Without this file it only knows about its built-in cloud providers.\n\nPlace this at `~/.config/docker-sbx/pi-llamacpp/files/home/.pi/agent/models.json`:\n\n```json\n{\n  \"providers\": {\n    \"llamacpp\": {\n      \"baseUrl\": \"http://host.docker.internal:8001/v1\",\n      \"api\": \"openai-completions\",\n      \"apiKey\": \"local-llama\",\n      \"models\": [\n        {\n          \"id\": \"unsloth/gemma-4-12B-it-qat-GGUF\",\n          \"name\": \"unsloth/gemma-4-12B-it-qat-GGUF\",\n          \"input\": [\"text\"]\n        }\n      ]\n    }\n  }\n}\n```\n\nThe `id` must exactly match the `--alias` value passed to llama-server. The\n`baseUrl` uses `host.docker.internal` for the reason explained in the\nArchitecture section.\n\n---\n\n## Step 7 — Create settings.json\n\nPi creates `~/.pi/agent/settings.json` on first launch, but only writes\ninternal state into it (`lastChangelogVersion` and similar). It does not set\n`defaultProvider` or `defaultModel`, so without this file in the kit Pi will\nprompt you to select a provider and model interactively on every session.\n\nBy injecting `settings.json` via the kit's `files/` directory, Pi picks up\nthe defaults before it runs for the first time and never asks.\n\nPlace this at `~/.config/docker-sbx/pi-llamacpp/files/home/.pi/agent/settings.json`:\n\n```json\n{\n  \"defaultProvider\": \"llamacpp\",\n  \"defaultModel\": \"unsloth/gemma-4-12B-it-qat-GGUF\"\n}\n```\n\n---\n\n## Step 8 — Run the sandbox\n\nFrom the project directory you want Pi to work in:\n\n```bash\nsbx run --kit ~/.config/docker-sbx/pi-llamacpp pi\n```\n\nOn first run sbx creates the microVM, injects `files/home/` into `~/` inside\nthe VM, and runs the npm install. This takes about 30-60 seconds. On\nsubsequent runs it reattaches to the existing sandbox instantly.\n\n\u003e **Important:** The `files/` injection only happens at sandbox creation time.\n\u003e If you modify `models.json` or `settings.json` after the sandbox already\n\u003e exists, destroy it and recreate it:\n\u003e\n\u003e ```bash\n\u003e sbx rm \u003csandbox-name\u003e\n\u003e sbx run --kit ~/.config/docker-sbx/pi-llamacpp pi\n\u003e ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcuolm%2Fpi-sbx-llamacpp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcuolm%2Fpi-sbx-llamacpp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcuolm%2Fpi-sbx-llamacpp/lists"}