{"id":28957973,"url":"https://github.com/walteh/runm","last_synced_at":"2025-10-11T21:09:01.140Z","repository":{"id":299166020,"uuid":"1001473678","full_name":"walteh/runm","owner":"walteh","description":"experimental macOS native container runtime 🍎🐧","archived":false,"fork":false,"pushed_at":"2025-08-19T13:08:19.000Z","size":9461,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-19T15:25:45.012Z","etag":null,"topics":["containerd","containers","go","golang","linux","macos","runc","virtulization","vm"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/walteh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-13T12:48:09.000Z","updated_at":"2025-08-19T13:08:23.000Z","dependencies_parsed_at":"2025-07-06T19:29:56.094Z","dependency_job_id":"42496a07-6f4b-45fc-b0b5-504faca4cd6c","html_url":"https://github.com/walteh/runm","commit_stats":null,"previous_names":["walteh/runv","walteh/runm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/walteh/runm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/walteh%2Frunm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/walteh%2Frunm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/walteh%2Frunm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/walteh%2Frunm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/walteh","download_url":"https://codeload.github.com/walteh/runm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/walteh%2Frunm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279008725,"owners_count":26084494,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["containerd","containers","go","golang","linux","macos","runc","virtulization","vm"],"created_at":"2025-06-23T22:35:33.382Z","updated_at":"2025-10-11T21:09:01.134Z","avatar_url":"https://github.com/walteh.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# runm\n\nExperimental per-container VM adaptor for `runc` that runs Linux containers on macOS by delegating Linux-specific operations to a tiny guest VM.\n\n\u003e [!WARNING]\n\u003e These docs are largely under-construction. They serve primarily as a way to organize my own thoughts. If you are interested in learning more, please contact me directly.\n\n## High-level notes\n\n-   This project exploring how far a native macOS `containerd` stack can go by offloading Linux-only behavior to a guest VM.\n-   Think of runm as “`runc` over vsock”: the Linux-only bits run inside a tiny VM - everything else remains standard `containerd` plumbing on the host.\n-   The design mirrors industry patterns (Kata Containers; Apple’s Containerization) while integrating with `containerd` and `nerdctl` natively on macOS.\n\n## Current status and limitations\n\n-   `nerdctl run/exec` support is solid and is by far the most well-tested\n-   `containerd`'s `native-snapshotter` currently requires a FUSE workaround via `bindfs`.\n-   BuildKit and multi-container/pod semantics are not implemented here yet (e.g., a K8s “pod” would require grouping multiple containers into one VM, Kata-style).\n\n### Commands / Frontend\n\n-   ✅ `nerdctl run` and `nerdctl exec`\n-   ✅ `nerdctl` container management\n-   ⚠️ `nerdctl build`: buildkit integration works, not much else\n-   🚧 `ctr`: untested\n-   🚧 `kubectl`: untested, needs pod support\n\n### Linux-only binaries running on macOS\n\n-   ✅ `containerd`\n-   ✅ `nerdctl`\n-   ✅ `buildctl`\n-   🚧 `buildkitd`: it runs and functions; I haven't completed a successful test yet\n\n#### `nerdctl run/exec` specifics\n\n-   [x] exit status returned to the host\n-   [x] bind mounts\n-   [x] read-only mounts (via `ro=true`)\n-   [x] `-d` detached mode\n-   [x] internet access (e.g., reach google.com)\n-   [x] `-it` interactive mode\n-   [x] `-e` environment variables\n-   [x] `-w` working directory\n-   [x] `-v` volumes\n-   [x] `-p` ports\n-   [x] `--tty` pty passthrough\n-   [] `-u` user\n\n## Quickstart (example)\n\n#### Prereqs\n\n-   `macFUSE` (kext)\n-   `bindfs`\n-   `Virtualization.framework`\n-   `docker`\n-   `go`\n-   `iTerm2` (Terminal.app also works, but all logs are enhanced for iTerm)\n\n#### steps\n\nThis will run a scenario that creates a container via `nerdctl run -d`, then runs a command via `nerdctl exec` that streams stdio back to the host at 1s intervals.\n\n\u003e [!NOTE]\n\u003e your password may be required to clean up any processes that have leaked.\n\n-   Clone this repo\n-   install all required forks to `../` (`go tool task fork:install:all`)\n-   Open two panes in iTerm2\n-   In the first pane, run `go tool task dev:containerd`\n-   Take a break, it will take a while on the first run to build the kernel\n-   Wait until `containerd` starts and the logs stop\n-   In the second pane, run `go tool task dev:2025-07-05:01`\n-   The second pane will show user-facing logs streaming from the demo container\n\n## Forks and diffs\n\nThere are many forks that are required to run this project in its current state. The changes are a mix of required logical changes and (mostly) debugging. Until I have some time to put more TLC into them, I will lay out the important logical changes and what they enable.\n\n\u003e [!NOTE]\n\u003e current nerdctl and BuildKit logic indirectly assume their binaries are built on the same OS as the container runtime. The \"[bug]\" notes below refer to this assumption—even though it's not truly an upstream bug.\n\nLast upstream sync: `2025-08-06`\n\n1. `containerd` ([diff](https://github.com/containerd/containerd/compare/main...walteh:main))\n\n    - **[bind mounts]** add `darwin` build support for the `core/mount` package by invoking `bindfs` via `exec.Cmd`\n\n    - **[rootless]** ignore `EPERM` from `Lchown`/`Chown`\n\n1. `nerdctl` ([diff](https://github.com/containerd/nerdctl/compare/main...walteh:main))\n\n    - **[bug]** on `darwin/arm64`, use `oci.WithDefaultSpecForPlatform(\"linux/arm64\")` when creating the OCI spec to prevent various \"not supported\" errors\n\n    - **[bug]** refactor the mount parsing logic to use Linux-specific logic on macOS\n\n1. `buildkit` ([diff](https://github.com/moby/buildkit/compare/master...walteh:master))\n\n    - **[bug]** on `darwin/arm64`, generate OCI spec with explicit platform: `GenerateSpecWithPlatform(ctx, nil, \"linux/arm64\", ...)` to avoid missing `.Process.Args` inside the OCI spec\n\n1. `fsutil` ([diff](https://github.com/tonistiigi/fsutil/compare/master...walteh:main))\n\n    - **[rootless]** ignore `EPERM` from `Lchown`/`Chown`\n\n1. `gvisor-tap-vsock` ([diff](https://github.com/containers/gvisor-tap-vsock/compare/main...walteh:main))\n\n    - **[feature]** add raw `net.Listener` port-forwarding support\n\n    - **[context/reliability]** pass context through missing places; return `ctx.Err()` on cancellation\n\n## How it works (high level)\n\n-   Host: a `containerd` shim on macOS\n-   Guest: a tiny Linux VM runs unmodified `runc` over gRPC/vsock\n-   IO/control: stdio, signals, exit codes flow over vsock\n-   Isolation: one micro‑VM per container (no shared kernel)\n-   Networking: per‑VM networking via gvisor‑tap‑vsock\n\nTo `containerd`, it looks like a normal OCI runtime; Linux syscalls execute inside the guest.\n\n**Rough outline**\n\n```mermaid\nsequenceDiagram\n    participant U as nerdctl\n    participant CD as containerd\n    participant SH as runm shim (host)\n    participant VF as Virtualization.framework\n    participant VM as runm guest (VM)\n    participant R as runc\n    participant P as container process\n\n    U-\u003e\u003eCD: run\n    CD-\u003e\u003eSH: create/start task\n    SH-\u003e\u003eVF: boot micro-VM\n    VF-\u003e\u003eSH: VM ready\n    SH-\u003e\u003eVM: gRPC Create/Start\n    VM-\u003e\u003eR: runc create/start\n    R-\u003e\u003eP: exec process\n    P-\u003e\u003eVM: stdout/stderr, exit\n    VM-\u003e\u003eSH: IO and status\n    SH-\u003e\u003eCD: task state\n    CD-\u003e\u003eU: result\n```\n\n---\n\n\u003e [!IMPORTANT]\n\u003e The below docs are even more so under construction and incomplete.\n\n## Unfinished notes / documentation\n\n#### Why\n\nmacOS does not provide Linux namespaces, cgroups, or overlayfs. Traditional solutions (e.g., Docker Desktop) run a single Linux VM hosting all containers. Runm explores a Kata-like “VM-per-container” design on macOS using Apple’s Virtualization.framework: stronger isolation, clean lifecycles, and a native integration path. The approach aligns with Apple’s Containerization framework direction while remaining OCI- and containerd-oriented.\n\n#### Architecture (deeper dive)\n\n-   Host shim: adapted from `containerd`’s `containerd-shim-runc-v2`. It translates container lifecycle requests into gRPC calls to the guest.\n-   Guest agent: receives requests over vsock and runs `runc` inside the VM. No changes to `runc` are required beyond extra debug.\n-   IO and control: stdio, signals, and exit codes traverse vsock; networking for the guest is provided via gvisor-tap-vsock style forwarding.\n\n#### Relationship to Kata Containers and Apple’s Containerization\n\n-   Like Kata Containers, runm boots a minimal guest and runs the workload inside that VM for stronger isolation.\n-   Like Apple’s Containerization, runm embraces one-VM-per-container on macOS using Virtualization.framework for fast boots and tight integration.\n-   Unlike Kata’s K8s-focused “one VM per pod” model, runm currently treats each `nerdctl run` as its own sandbox VM.\n\n#### Linux\n\nRunm offloads Linux operations to a lightweight VM launched with Virtualization.framework.\n\n-   Custom Linux kernel configuration lives in `./linux/kernel`.\n-   Static `busybox` is built in `./linux/busybox`.\n\nFile system layout:\n\n```\n# initramfs\n/init (symlink to `/runm-linux-mounter`)\n/runm-linux-mounter\n/bin/busybox\n```\n\n`runm-linux-mounter` is the only binary in the initramfs; it mounts the `mbin` squashfs containing the remaining guest binaries.\n\n```\n# rootfs\n/bin/busybox\n/mbin/runm-linux-init\n/mbin/runc-test\n/mbin/runm-linux-host-fork-exec-proxy\n```\n\n#### Snapshotting and macFUSE\n\nOn Linux, `containerd`’s native snapshotter relies on bind mounts (`mount --bind`). macOS has no `mount --bind`, so we use `bindfs` and a FUSE implementation to simulate bind mounts:\n\n-   `bindfs` (OSS, GPL-2) + either `macFUSE` or `fuse-t` (free to use, closed-source) enable host-side bind-like behavior for the native snapshotter.\n-   `macFUSE` kext requires reduced security mode; `fuse-t` avoids a kext but has proven unstable in practice.\n-   macOS 15 introduced FSKit for user-space file systems; `macFUSE` v5 advertises support, but it has not worked out-of-the-box here yet.\n\nNotes from experimentation:\n\n-   `fuse-t` was significantly less stable (e.g., sporadic missing `glibc` files breaking dynamic linking).\n-   `macFUSE (kext)` has been much more reliable.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwalteh%2Frunm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwalteh%2Frunm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwalteh%2Frunm/lists"}