{"id":47259501,"url":"https://github.com/devigned/containerd-cloudhypervisor","last_synced_at":"2026-04-01T22:21:37.471Z","repository":{"id":343023635,"uuid":"1175445179","full_name":"devigned/containerd-cloudhypervisor","owner":"devigned","description":"A svelte containerd shim for running VM isolated containers using Cloud Hypervisor","archived":false,"fork":false,"pushed_at":"2026-03-21T21:47:11.000Z","size":814,"stargazers_count":0,"open_issues_count":4,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-22T10:46:45.706Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/devigned.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-07T18:10:52.000Z","updated_at":"2026-03-21T21:47:01.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/devigned/containerd-cloudhypervisor","commit_stats":null,"previous_names":["devigned/containerd-cloudhypervisor"],"tags_count":38,"template":false,"template_full_name":null,"purl":"pkg:github/devigned/containerd-cloudhypervisor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devigned%2Fcontainerd-cloudhypervisor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devigned%2Fcontainerd-cloudhypervisor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devigned%2Fcontainerd-cloudhypervisor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devigned%2Fcontainerd-cloudhypervisor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/devigned","download_url":"https://codeload.github.com/devigned/containerd-cloudhypervisor/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devigned%2Fcontainerd-cloudhypervisor/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31292639,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T21:15:39.731Z","status":"ssl_error","status_checked_at":"2026-04-01T21:15:34.046Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-03-14T22:29:25.583Z","updated_at":"2026-04-01T22:21:37.461Z","avatar_url":"https://github.com/devigned.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# containerd-cloudhypervisor\n\nA purpose-built [containerd](https://containerd.io/) shim for [Cloud Hypervisor](https://www.cloudhypervisor.org/)\nthat runs container workloads inside lightweight microVMs with maximum density and minimal memory overhead.\n\n## Highlights\n\n- **Sandbox daemon** — long-running systemd daemon pre-boots a pool of VMs from a base snapshot via CH v51 OnDemand restore (~25ms each, ~6 MB idle RSS). Shadow VMs create warm workload snapshots in the background — no production pod pausing\n- **150/150 pods in 11s** on 3 × D8ds_v5 nodes (96 GiB total RAM), ~24 MB per VM, 10% node memory utilization\n- **Thin shim** — the containerd shim (~1,300 lines) handles TAP networking, erofs conversion, and daemon RPCs (`AcquireSandbox`, `AddContainer`, `ReleaseSandbox`)\n- **VM isolation** — each pod runs in its own Cloud Hypervisor microVM with dedicated kernel\n- **erofs rootfs cache** — content-addressable, flock-serialized, shared across pods\n- **inotify device discovery** — hot-plugged container disks detected in \u003c1ms via inotify (no polling)\n- **Pure libc networking** — TAP/tc setup via in-process netlink (\u003c1ms, no subprocess)\n- **Dual hypervisor** — same binary runs on KVM (Linux) and MSHV (Azure/Hyper-V)\n- **Multi-container pods** — up to 5 containers per VM with mount + PID isolation\n- **Pod networking** — transparent CNI integration via TAP + TC redirect\n- **Kata-compatible annotations** — per-pod memory/vCPU sizing with `io.cloudhv.*` or `io.katacontainers.*`\n- **Transparent vCPU sizing** — VM vCPUs match the pod's CPU limit; no limit = host CPU count\n\n## When to Use\n\nChoose this shim when you're building a **platform** where you control the stack and need\nVM isolation without the overhead of a full-featured VMM stack. Ideal for AI agent sandboxes,\nserverless/FaaS platforms, and security-sensitive workloads where density matters.\n\nFor general-purpose Kubernetes with multi-hypervisor support, GPU passthrough, or live\nmigration, consider [Kata Containers](https://katacontainers.io/) instead.\n\n| | containerd-cloudhypervisor | Kata Containers |\n| --- | --- | --- |\n| **Cold start (shim inner)** | ~74ms | ~500ms–1s |\n| **Warm restore** | ~168ms | N/A |\n| **Memory per pod** | ~24 MB (OnDemand CoW) | ~330 MB |\n| **150-pod scale** | 150/150 in 11s | 130/150 (OOM) |\n| **Shim binary** | 4.6 MB | ~50 MB |\n| **Guest rootfs** | 5.4 MB (agent + crun, erofs) | ~150 MB |\n| **Language** | Rust | Go |\n\n## Quick Start\n\n### System extension for Flatcar and similar container OSes\n\nA self-contained system extension image is shipped with each [release](releases/); there's a Butane snippet included with the release notes for provisioning the extension.\nThe general pattern is\n```\nvariant: flatcar\nversion: 1.0.0\n\nstorage:\n  files:\n  - path: /etc/extensions/containerd-cloudhypervisor.raw\n    mode: 0644\n    contents:\n      source: https://github.com/devigned/containerd-cloudhypervisor/releases/download/\u003crelease-version\u003e/containerd-cloudhypervisor-\u003crelease-version\u003e-x86-64.raw\n```\n\nThe sysext includes a brief demo to verify if the system is working. Run\n```shell\nroot@flatcar $ /usr/share/cloudhv/demo/demo.sh\n```\nto verify.\n\n#### Test your builds locally in a Flatcar VM\n\nSysext integration makes it easy to build the repository and run it locally in a Flatcar VM.\n\nFirst, build the sysext.\nThis build is containerised and has no host dependencies (except Docker).\n```\nbash hacks/build-sysext.sh\n```\n\nFor local testing, we'll leverage the [`boot` feature](https://github.com/flatcar/sysext-bakery?tab=readme-ov-file#interactively-test-extension-images-in-a-local-vm)\nof Flatcar's [sysext bakery](https://github.com/flatcar/sysext-bakery).\n\n1. Check out the bakery repo into a separate directory:\n   ```\n   git clone --depth 1 https://github.com/flatcar/sysext-bakery.git\n   ```\n2. Copy `containerd-cloudhypervisor.raw` into the bakery repo root; change into the bakery repo root.\n3. Run\n   ```\n   ./bakery.sh boot containerd-cloudhypervisor.raw\n   ```\n\nThis will download the latest Flatcar Alpha release for qemu, then start a Flatcar VM in ephemeral mode (no changes will be persisted in the Flatcar OS image).\n`bakery.sh boot` will also launch a local Python webserver and generate transient Ignition configuration to provision `containerd-cloudhypervisor.raw` at boot time.\n\nAfter the VM boot finished, you'll end up on the VM's serial port.\nRun the demo included with the extension image to verify:\n```bash\nsudo /usr/share/cloudhv/demo/demo.sh\n```\n\nYou can also connect to the local VM via ssh, using the `core` user:\n```bash\nssh -p 2222 core@localhost\n```\n\n### Manual installation\n\n```bash\n# Build\ncargo build --release -p containerd-shim-cloudhv\ncargo build --release -p cloudhv-sandbox-daemon\ncargo build --release -p cloudhv-agent --target x86_64-unknown-linux-musl\ncd guest/kernel \u0026\u0026 bash build-kernel.sh \u0026\u0026 cd ../..\ncd guest/rootfs \u0026\u0026 sudo bash build-rootfs.sh ../../target/x86_64-unknown-linux-musl/release/cloudhv-agent \u0026\u0026 cd ../..\n\n# Install binaries\nsudo install -m 755 target/release/containerd-shim-cloudhv-v1 /usr/local/bin/\nsudo install -m 755 target/release/cloudhv-sandbox-daemon /usr/local/bin/\nsudo mkdir -p /opt/cloudhv /run/cloudhv/erofs-cache /run/cloudhv/daemon\nsudo cp guest/kernel/vmlinux guest/rootfs/rootfs.erofs /opt/cloudhv/\n\n# Shim config (see docs/configuration.md for full reference)\nsudo tee /opt/cloudhv/config.json \u003e /dev/null \u003c\u003cEOF\n{\n  \"cloud_hypervisor_binary\": \"/usr/local/bin/cloud-hypervisor\",\n  \"kernel_path\": \"/opt/cloudhv/vmlinux\",\n  \"rootfs_path\": \"/opt/cloudhv/rootfs.erofs\",\n  \"kernel_args\": \"console=ttyS0 root=/dev/vda rw init=/init net.ifnames=0\",\n  \"default_vcpus\": 1,\n  \"max_default_vcpus\": 0,\n  \"default_memory_mb\": 128,\n  \"max_containers_per_vm\": 5,\n  \"daemon_socket\": \"/run/cloudhv/daemon.sock\"\n}\nEOF\n\n# Daemon config\nsudo tee /opt/cloudhv/daemon.json \u003e /dev/null \u003c\u003cEOF\n{\n  \"pool_size\": 3,\n  \"max_pool_size\": 10,\n  \"default_vcpus\": 1,\n  \"default_memory_mb\": 128,\n  \"kernel_path\": \"/opt/cloudhv/vmlinux\",\n  \"rootfs_path\": \"/opt/cloudhv/rootfs.erofs\",\n  \"kernel_args\": \"console=ttyS0 root=/dev/vda rw init=/init net.ifnames=0\",\n  \"socket_path\": \"/run/cloudhv/daemon.sock\",\n  \"state_dir\": \"/run/cloudhv/daemon\",\n  \"warmup_duration_secs\": 30,\n  \"max_snapshots\": 100\n}\nEOF\n\n# Systemd unit for the daemon\nsudo tee /etc/systemd/system/cloudhv-sandbox-daemon.service \u003e /dev/null \u003c\u003cEOF\n[Unit]\nDescription=CloudHV Sandbox Daemon\nAfter=containerd.service\nRequires=containerd.service\n\n[Service]\nType=simple\nExecStartPre=/bin/mkdir -p /run/cloudhv/daemon\nExecStart=/usr/local/bin/cloudhv-sandbox-daemon /opt/cloudhv/daemon.json\nRestart=always\nRestartSec=5\nEnvironment=RUST_LOG=info\nMemoryMax=4G\n\n[Install]\nWantedBy=multi-user.target\nEOF\n\nsudo systemctl daemon-reload\nsudo systemctl enable --now cloudhv-sandbox-daemon\n```\n\n## Documentation\n\nSee the **[docs/](docs/)** folder for detailed documentation:\n\n- **[Architecture](docs/architecture.md)** — system design, daemon + shim + agent components, networking\n- **[Sandbox Daemon](docs/sandbox-daemon.md)** — daemon design, VM pool, shadow snapshots, benchmark results\n- **[Configuration](docs/configuration.md)** — shim and daemon config reference, pod annotations\n- **[Performance](docs/performance.md)** — benchmarks, latency breakdown, density, comparison with Kata\n- **[Development](docs/development.md)** — building, testing, contributing, code quality standards\n- **[Releasing](docs/releasing.md)** — release workflow, published artifacts, installation\n\n## Examples\n\n- **[Bare Linux with crictl](example/crictl/)** — run containers with crictl, no Kubernetes required\n- **[Azure Kubernetes Service](example/aks/)** — deploy on AKS with DaemonSet installer\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevigned%2Fcontainerd-cloudhypervisor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevigned%2Fcontainerd-cloudhypervisor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevigned%2Fcontainerd-cloudhypervisor/lists"}