{"id":51573119,"url":"https://github.com/czhao-dev/container-runtime","last_synced_at":"2026-07-10T21:30:35.554Z","repository":{"id":370081470,"uuid":"1287770565","full_name":"czhao-dev/container-runtime","owner":"czhao-dev","description":"A minimal container runtime CLI in Go: Linux namespaces, pivot_root filesystem jailing, cgroups v2 resource limits, and interactive PTY passthrough — no image management, overlayfs, or daemon.","archived":false,"fork":false,"pushed_at":"2026-07-08T07:11:56.000Z","size":66,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-08T07:17:02.698Z","etag":null,"topics":["cgroups","cli","container-runtime","containers","go","golang","linux","namespaces","pivot-root"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/czhao-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-03T01:55:55.000Z","updated_at":"2026-07-08T07:11:59.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/czhao-dev/container-runtime","commit_stats":null,"previous_names":["czhao-dev/container-runtime"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/czhao-dev/container-runtime","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czhao-dev%2Fcontainer-runtime","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czhao-dev%2Fcontainer-runtime/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czhao-dev%2Fcontainer-runtime/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czhao-dev%2Fcontainer-runtime/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/czhao-dev","download_url":"https://codeload.github.com/czhao-dev/container-runtime/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czhao-dev%2Fcontainer-runtime/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35344523,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-10T02:00:06.465Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cgroups","cli","container-runtime","containers","go","golang","linux","namespaces","pivot-root"],"created_at":"2026-07-10T21:30:34.817Z","updated_at":"2026-07-10T21:30:35.539Z","avatar_url":"https://github.com/czhao-dev.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mini-run\n\nA minimal container runtime CLI in Go — the core primitives real runtimes\n(runc) use, without image management, overlayfs, a daemon, or networking:\n\n- **Namespace isolation** — `CLONE_NEWPID` (container sees itself as PID 1),\n  `CLONE_NEWNS` (isolated mount table), `CLONE_NEWUTS` (custom hostname),\n  `CLONE_NEWIPC` (isolated IPC), via `syscall.SysProcAttr.Cloneflags`.\n- **Filesystem jailing via `pivot_root`** — not `chroot`: the old root is\n  fully unmounted and detached, preventing container breakout.\n- **cgroups v2 resource limits** — per-container memory ceiling and CPU\n  quota under `/sys/fs/cgroup/mini-run/\u003cid\u003e/`.\n- **Interactive PTY passthrough** — stdio is wired straight through so\n  `/bin/sh`, `top`, etc. work interactively.\n\nIt is a stateless CLI, not a daemon: `sudo mini-run run \u003crootfs\u003e \u003ccmd\u003e [args...]`\nagainst a rootfs directory you've already unpacked yourself (e.g. Alpine's\nminirootfs tarball, extracted).\n\n## Repository Layout\n\n```\n.\n├── cmd/mini-run/             CLI entrypoint\n│   ├── main.go               arg dispatch: run | child (hidden) | -h\n│   ├── run.go                `run` subcommand: flags -\u003e container.RunOptions\n│   └── child.go              hidden `child` subcommand -\u003e container.RunChild\n├── internal/\n│   ├── container/\n│   │   ├── types.go          RunOptions\n│   │   ├── run_linux.go      parent-side orchestration (namespaces, cgroups, wait)\n│   │   ├── child_linux.go    PID-1 child logic: hostname, pivot_root, exec\n│   │   ├── rootfs_linux.go   pivot_root + synthetic /dev implementation\n│   │   ├── id.go             random 12-hex-char container ID\n│   │   ├── id_test.go\n│   │   └── *_other.go        non-Linux build-tag stubs\n│   └── cgroups/\n│       ├── cgroups_linux.go  cgroups v2 Setup / AddProcess / Cleanup\n│       ├── limits.go         ParseMemory / ParseCPUs (pure, cross-platform)\n│       ├── limits_test.go\n│       └── cgroups_other.go  non-Linux build-tag stub\n├── test/integration/\n│   └── mini_run_test.go      black-box, root+Linux-gated integration suite\n│                              (automates the Verification checklist below)\n├── scripts/\n│   ├── fetch-rootfs.sh       downloads + extracts Alpine's minirootfs\n│   ├── bench.sh              startup-latency benchmark harness (hyperfine)\n│   └── plot_bench.py         renders the benchmark bar chart (matplotlib)\n├── docs/\n│   └── benchmark.png         chart embedded in this README\n├── lima.yaml                 disposable Ubuntu VM config for macOS dev/test\n└── go.mod                    zero external dependencies\n```\n\n## Architecture\n\n`mini-run` has no long-running daemon and no supervisor process tree: the\nparent CLI invocation sets up resource limits, re-execs itself into new\nnamespaces, and then gets out of the way — the containerized command\nbecomes a real PID 1, not a child of some Go supervisor loop.\n\n```mermaid\nsequenceDiagram\n    participant Host as Host shell\n    participant Parent as mini-run run (parent)\n    participant CG as cgroups v2 (/sys/fs/cgroup/mini-run/\u003cid\u003e)\n    participant Child as mini-run child (becomes PID 1)\n    participant Target as exec'd command\n\n    Host-\u003e\u003eParent: sudo mini-run run --memory 128m ./rootfs /bin/sh\n    Parent-\u003e\u003eParent: NewID(), resolve rootfs, ParseLimits()\n    Parent-\u003e\u003eCG: mkdir \u003cid\u003e/, write memory.max / cpu.max\n    Parent-\u003e\u003eChild: re-exec self as \"child\" with Cloneflags\u003cbr/\u003eNEWNS|NEWUTS|NEWIPC|NEWPID, Pdeathsig=SIGKILL\n    Note over Child: new process, born as PID 1 inside\u003cbr/\u003eits own PID/mount/UTS/IPC namespaces\n    Parent-\u003e\u003eCG: write cgroup.procs (attach child's real host PID)\n    Child-\u003e\u003eChild: Sethostname(\"mini-run-\u003cid[:8]\u003e\")\n    Child-\u003e\u003eChild: pivotRoot(rootfs) — see filesystem diagram below\n    Child-\u003e\u003eTarget: syscall.Exec(resolved cmd) — replaces its own process image\n    Note over Target: target command IS the container's PID 1 now,\u003cbr/\u003estdio was wired straight through from the host shell\n    Target--\u003e\u003eParent: process exits\n    Parent-\u003e\u003eCG: rmdir \u003cid\u003e/ (Cleanup(), retries up to ~500ms)\n    Parent--\u003e\u003eHost: propagate the container's exit code\n```\n\nTwo design choices worth calling out:\n\n- **`syscall.Exec`, not a subprocess** — the child process replaces its own\n  image with the target command instead of spawning it as a child. This\n  means the container's PID 1 *is* `/bin/sh` (or whatever was requested),\n  with correct init/signal/reaping semantics, rather than a Go process that\n  has to babysit a subprocess.\n- **cgroup limits are created before the container exists** — `memory.max`\n  and `cpu.max` don't require a PID to be written, so the cgroup is fully\n  configured first and the container's host PID is attached to it right\n  after the namespaced process starts, closing the window where an\n  unconstrained process could run free.\n\n### Filesystem jailing: `pivot_root`, not `chroot`\n\n`chroot` only changes what path resolution treats as `/`; the old root\nfilesystem is still mounted and reachable through tricks like relative\n`..` traversal via an open file descriptor. `pivot_root` swaps the entire\nmount tree's root and lets the old root be fully unmounted and deleted, so\nthere is nothing left to break out into:\n\n```\nBefore pivot_root                      After pivot_root\n(inside the new mount namespace)       (inside the new mount namespace)\n----------------------------------     ----------------------------------\n/                                      /                                   (== former rootfs,\n├── proc/, sys/, dev/, ...             ├── bin/, etc/, usr/, ...           now the real root)\n├── home/.../container-runtime/        ├── dev/                            (fresh tmpfs: null, zero,\n│   └── rootfs/  \u003c- bind-mounted       │                                   full, random, urandom,\n│       onto itself so pivot_root(2)   │                                   tty, fd -\u003e /proc/self/fd)\n│       accepts it as the new root     ├── proc/                           (freshly mounted here,\n└── ...rest of the host filesystem...  │                                   reflects the new PID ns)\n                                       └── .old_root                       unmounted (MNT_DETACH) and\n                                                                           os.RemoveAll'd -- gone\n```\n\nThe sequence, from [`rootfs_linux.go`](internal/container/rootfs_linux.go):\nmake all mounts private+recursive (so `pivot_root` doesn't `EINVAL` on\ndistros with shared mount propagation) → bind-mount `rootfs` onto itself\n(a mount-point requirement of `pivot_root(2)`) → build a minimal synthetic\n`/dev` via tmpfs + hand-rolled `mknod` (never bind-mount the host's real\n`/dev`) → `pivot_root` → `chdir(\"/\")` → mount a fresh `/proc` (only valid\n*after* the pivot, since it must reflect the new PID namespace) → unmount\nand recursively delete `.old_root`.\n\n## Requirements\n\nNamespaces, `pivot_root`, and cgroups v2 are Linux kernel features and do\nnot exist on macOS/Windows. Building the CLI works anywhere (Go's build\ntags isolate the Linux-only code), but actually **running** a container\nrequires Linux, root, and a cgroups v2 unified hierarchy mounted at\n`/sys/fs/cgroup`.\n\n### Dev/test environment (macOS)\n\nA [Lima](https://lima-vm.io/) config is included for a disposable Ubuntu VM:\n\n```\nbrew install lima\nlimactl start --name mini-run ./lima.yaml\nlimactl shell mini-run\ncd container-runtime\ngo build -o mini-run ./cmd/mini-run\n./scripts/fetch-rootfs.sh\nsudo ./mini-run run --memory 128m --cpus 0.5 ./rootfs /bin/sh\n```\n\n(Ubuntu/systemd is used rather than Alpine/OpenRC because systemd keeps the\nroot cgroup free of directly-attached processes — otherwise enabling the\ncpu/memory controllers on `/sys/fs/cgroup` fails with `EBUSY`.)\n\nIf you already have a Linux box, skip Lima and just `go build` +\n`sudo ./mini-run run ...` there directly.\n\n## Usage\n\n```\nsudo mini-run run [--memory 512m] [--cpus 0.5] \u003crootfs-path\u003e \u003ccmd\u003e [args...]\n```\n\n- `--memory` — e.g. `512m`, `1.5g` (default: unlimited)\n- `--cpus` — fraction of one core, e.g. `0.5`, `2` (default: unlimited)\n\nFlags must precede `\u003crootfs-path\u003e`; everything after the command is passed\nthrough untouched (so `sudo mini-run run ./rootfs /bin/sh -c \"ls -la\"` works\nas expected).\n\n## Verification\n\nRun inside the Lima VM (or any Linux box):\n\n1. **PID namespace**: `echo $$` inside the container prints `1`.\n2. **UTS namespace**: `hostname` prints a generated `mini-run-xxxxxxxx` name,\n   distinct from the host's.\n3. **Mount namespace**: `mount` inside the container shows only `/`,\n   `/proc`, and the synthetic `/dev` tmpfs — compare against a second\n   `limactl shell mini-run` window on the host.\n4. **Process visibility**: `ps aux` inside the container shows only the\n   container's own processes, not the host's (`/proc` reflects the new PID\n   namespace).\n5. **Breakout prevention**: `ls /.old_root` inside the container fails with\n   \"No such file or directory\"; `ls /` shows only the rootfs's own\n   directories, never host paths like `/home` or the project directory.\n6. **cgroup limits applied**: from the host, `cat /sys/fs/cgroup/mini-run/\u003cid\u003e/memory.max`\n   and `.../cpu.max` match what was requested; `cgroup.procs` shows the real\n   host-namespace PID (vs. `1` seen inside the container).\n7. **Memory limit enforced**: inside the container, run a memory bomb\n   (`sh -c 'a=AAAA; while true; do a=\"$a$a\"; done'`) — it gets OOM-killed by\n   the cgroup, without affecting the host VM.\n8. **CPU limit enforced**: `yes \u003e /dev/null` inside the container while\n   watching host-side `top` — usage caps near the configured fraction of one\n   core.\n9. **Cleanup**: after `exit` (or Ctrl-C mid-session), `/sys/fs/cgroup/mini-run/\u003cid\u003e`\n   is gone.\n10. **Exit code propagation**: `sudo ./mini-run run ./rootfs /bin/sh -c \"exit 42\"; echo $?`\n    prints `42`.\n\nThe portable pieces (flag parsing, ID generation, memory/CPU string\nparsing) build and vet directly on macOS without Lima:\n\n```\ngo build ./... \u0026\u0026 go vet ./...\n```\n\n## Testing\n\nUnit tests cover the pure, cross-platform logic (ID generation, memory/CPU\nparsing) and run anywhere, no Lima or root needed:\n\n```\ngo test ./...\n```\n\nEverything else — namespace isolation, `pivot_root`, cgroup enforcement,\nexit-code propagation, cleanup — is covered by an integration suite that\nautomates the 10 verification steps above. It's black-box (drives the built\nbinary as a real user would) and requires Linux, root, and a fetched rootfs,\nso it's gated behind the `integration` build tag and skips itself with a\nclear message if either is missing. Run it inside the Lima VM:\n\n```\ngo build -o mini-run ./cmd/mini-run\n./scripts/fetch-rootfs.sh\nsudo go test -tags integration ./test/integration/... -v\n```\n\n## Benchmarking\n\n`scripts/bench.sh` measures container startup latency and compares it\nagainst a staircase of baselines to show what each isolation layer costs:\nraw exec, `chroot` alone, `chroot` + namespaces (via `unshare`), and\nmini-run itself (+ cgroups v2 setup/cleanup and `pivot_root`). It uses\n[hyperfine](https://github.com/sharkdp/hyperfine) for statistically sound\ntiming and, if `matplotlib` is available, renders `bench-results.png` (via\n`scripts/plot_bench.py`) as a labeled bar chart of the four variants. Run\ninside the Lima VM (both `hyperfine` and `matplotlib` are preinstalled by\n`lima.yaml`'s provisioning script):\n\n```\nsudo ./scripts/bench.sh\n```\n\n### Results\n\nMeasured on an aarch64 Lima VM (2 vCPUs, 4GiB RAM, Ubuntu 24.04) running on\na macOS host — absolute numbers will vary by machine, but the shape of the\nstaircase (what each layer adds) is the interesting part:\n\n![Container startup latency: raw exec vs. chroot vs. chroot+namespaces vs. mini-run](docs/benchmark.png)\n\n| Command | Mean | Min | Max | Slowdown vs. raw exec |\n|:---|---:|---:|---:|---:|\n| a: raw exec | 1.1 µs | 0.0 µs | 1027.1 µs | 1× |\n| b: chroot only | 253.4 µs | 166.1 µs | 5011.2 µs | ~238× |\n| c: chroot + namespaces | 519.8 µs | 452.4 µs | 1132.3 µs | ~488× |\n| d: mini-run | 3.36 ms | 2.16 ms | 17.98 ms | ~3160× |\n\n**Reading the staircase:**\n\n- **(a) raw exec** — `true`, no isolation at all. Hyperfine's own warning\n  (\"command took less than 5 ms\") is the tell that this number is really\n  measuring hyperfine's shell-invocation floor, not a meaningful process\n  cost — treat it as \"approximately zero,\" the baseline everything else is\n  relative to, not a precise figure.\n- **(a) → (b) chroot only, +252 µs** — this jump is *not* the cost of the\n  `chroot(2)` syscall itself (that's sub-microsecond). It's the cost of\n  spawning the external `chroot` binary as its own process before it execs\n  `/bin/true` — i.e. this measures \"one extra fork+exec plus a `chroot()`\n  and `chdir()` call,\" which is representative of what a shell-script-based\n  jail would actually cost, not a syscall microbenchmark in isolation.\n- **(b) → (c) + namespaces via `unshare`, +266 µs** — on top of another\n  process spawn (the `unshare` binary itself), the kernel now allocates and\n  populates a new PID, mount, UTS, and IPC namespace (`CLONE_NEWPID|\n  CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWIPC`). Namespace creation has real\n  kernel-side cost — the mount namespace in particular requires copying the\n  process's mount table — which is why this step costs meaningfully more\n  than the chroot-only step even though both are dominated by process-spawn\n  overhead.\n- **(c) → (d) mini-run, +2.84 ms** — this is where mini-run's extra work\n  over the `unshare`+`chroot` equivalent shows up, and it's the sum of\n  several things the manual baseline doesn't do at all:\n  1. **Two full process launches, not one `clone(2)`** — the parent\n     (`mini-run run`) re-execs *itself* as `mini-run child` via\n     `os/exec.Command` rather than calling `clone(2)` directly in-process.\n     That's two separate ~2.8MB Go binary startups (runtime init, GC init,\n     etc.) chained together, versus `unshare`'s single `clone()`+`execve()`.\n  2. **cgroups v2 setup and teardown** — `mkdir`, two `subtree_control`\n     writes (root and `mini-run/`), `memory.max`/`cpu.max` writes, a\n     `cgroup.procs` attach, and on the way out, an `rmdir` that retries up\n     to 10×50ms if kernel accounting hasn't caught up yet. Each is a\n     synchronous sysfs write with kernel-side validation — `unshare`/`chroot`\n     do none of this.\n  3. **The full `pivot_root` sequence** — making mounts private+recursive,\n     a recursive bind-mount of the rootfs onto itself, a tmpfs mount plus\n     six `mknod` calls for `/dev`, `pivot_root(2)` itself, and finally\n     unmounting (`MNT_DETACH`) and recursively deleting the old root.\n     `chroot` is a single syscall by comparison; this is roughly a dozen.\n  4. A fresh `/proc` mount after the pivot (namespace-aware, so it can't\n     happen before).\n  - The wide variance (±1.86 ms, up to 18 ms) is consistent with this: it's\n    a long chain of syscalls in a nested-virtualization guest (Lima VM on\n    top of macOS), plus the cgroup cleanup retry loop occasionally kicking\n    in if a previous container's accounting hadn't settled yet.\n\nThe takeaway: **mini-run's overhead over hand-rolled namespaces is almost\nentirely cgroups v2 bookkeeping and the `pivot_root`/`/dev` setup, not the\nnamespaces themselves** — namespace creation (b→c) is cheap relative to\nwhat a real container runtime does around it (c→d). This roughly tracks\nhow runc and similar runtimes spend their startup budget too.\n\n### Why this result is significant\n\nThe headline number — mini-run is ~3,000× slower than a bare `exec` — sounds\ndamning until it's put in context: it's 3.36 **milliseconds**. The point of\nthe staircase isn't \"containers are slow,\" it's *decomposing* a number that\nis usually reported as a single, opaque figure (\"container startup takes N\nms\") into the four architectural decisions that actually produce it. Once\ndecomposed, a more interesting fact emerges: **namespace creation itself\n(b→c) is nearly free** — a few hundred microseconds — while the bulk of the\ncost (c→d) comes from bookkeeping *around* the namespaces: cgroup sysfs\nI/O, an extra process re-exec, and filesystem jail setup. That's a useful,\ntransferable insight about where container overhead actually lives, and\nit's only visible because each layer was isolated and measured on its own\nrather than benchmarked as one monolithic `docker run`.\n\nIt's also evidence that the implementation is doing what it claims to: if\n`pivot_root`, cgroup limit enforcement, or namespace isolation were silently\nno-ops, the latency staircase wouldn't exist — each step costs measurably\nmore than the last specifically because each one does real, verifiable\nkernel work (independently confirmed by the [integration test suite](#testing)\nabove, which checks the *behavior*, not just the timing).\n\n## References\n\n**Primitives this project implements:**\n\n- [`namespaces(7)`](https://man7.org/linux/man-pages/man7/namespaces.7.html),\n  [`pid_namespaces(7)`](https://man7.org/linux/man-pages/man7/pid_namespaces.7.html),\n  [`mount_namespaces(7)`](https://man7.org/linux/man-pages/man7/mount_namespaces.7.html) —\n  Linux manual pages for the isolation primitives behind `CLONE_NEWPID` /\n  `CLONE_NEWNS` / `CLONE_NEWUTS` / `CLONE_NEWIPC`.\n- [`pivot_root(2)`](https://man7.org/linux/man-pages/man2/pivot_root.2.html) —\n  why it's used instead of `chroot(2)` for filesystem jailing.\n- [cgroups v2 (kernel docs)](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html) —\n  the unified hierarchy, `subtree_control`, `memory.max`, `cpu.max`.\n- [OCI Runtime Specification](https://github.com/opencontainers/runtime-spec) —\n  the spec real container runtimes (runc, crun) implement; mini-run\n  deliberately implements a subset of the same underlying primitives\n  without the spec's config/bundle format.\n- [runc](https://github.com/opencontainers/runc) — the reference OCI\n  runtime implementation; mini-run's `pivot_root` sequence follows the\n  same shape (private+recursive remount, self bind-mount, pivot, detach).\n\n**Tooling used in this repo:**\n\n- [Lima](https://lima-vm.io/) — the Linux VM used for macOS development\n  ([`lima.yaml`](lima.yaml)).\n- [hyperfine](https://github.com/sharkdp/hyperfine) — the benchmarking tool\n  used by [`scripts/bench.sh`](scripts/bench.sh).\n- [Alpine Linux `minirootfs`](https://alpinelinux.org/downloads/) — the\n  rootfs fetched by [`scripts/fetch-rootfs.sh`](scripts/fetch-rootfs.sh).\n\n## License\n\nThis project is licensed under the MIT License. See [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fczhao-dev%2Fcontainer-runtime","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fczhao-dev%2Fcontainer-runtime","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fczhao-dev%2Fcontainer-runtime/lists"}