{"id":13903248,"url":"https://github.com/pdtpartners/nix-snapshotter","last_synced_at":"2025-05-15T11:08:18.916Z","repository":{"id":193153999,"uuid":"662720293","full_name":"pdtpartners/nix-snapshotter","owner":"pdtpartners","description":"Brings native understanding of Nix packages to containerd","archived":false,"fork":false,"pushed_at":"2024-12-15T19:04:03.000Z","size":4687,"stargazers_count":642,"open_issues_count":23,"forks_count":20,"subscribers_count":15,"default_branch":"main","last_synced_at":"2025-04-14T19:59:17.589Z","etag":null,"topics":["container-image","containerd","containers","docker","docker-image","home-manager","nix","nixos","snapshotter"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pdtpartners.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-05T18:31:50.000Z","updated_at":"2025-04-12T17:19:36.000Z","dependencies_parsed_at":"2024-01-14T04:40:52.161Z","dependency_job_id":"4df49ceb-592d-457a-b63a-36a60f4595a1","html_url":"https://github.com/pdtpartners/nix-snapshotter","commit_stats":{"total_commits":79,"total_committers":8,"mean_commits":9.875,"dds":0.3291139240506329,"last_synced_commit":"6eaadfd8f89e5e7d79b2013626bbd36e388159da"},"previous_names":["pdtpartners/nix-snapshotter"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pdtpartners%2Fnix-snapshotter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pdtpartners%2Fnix-snapshotter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pdtpartners%2Fnix-snapshotter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pdtpartners%2Fnix-snapshotter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pdtpartners","download_url":"https://codeload.github.com/pdtpartners/nix-snapshotter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254328384,"owners_count":22052632,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["container-image","containerd","containers","docker","docker-image","home-manager","nix","nixos","snapshotter"],"created_at":"2024-08-06T22:01:54.569Z","updated_at":"2025-05-15T11:08:18.891Z","avatar_url":"https://github.com/pdtpartners.png","language":"Go","funding_links":[],"categories":["Virtualisation","nix","Go"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# nix-snapshotter\n\n[![Go Reference][go-reference-badge]][go-reference]\n[![ci][ci-badge]][ci]\n[![Go Report Card][go-report-card-badge]][go-report-card]\n\nBrings native understanding of [Nix][nix] packages to [containerd][containerd].\n\n[Key features](#key-features) •\n[Getting started](#getting-started) •\n[Installation](#installation) •\n[Architecture][architecture] •\n[Contributing](CONTRIBUTING.md)\n\n\u003c/div\u003e\n\n## Key features\n\n- Instead of downloading image layers, software packages come directly from a\n  Nix store.\n- Packages can be fetched from a Nix binary cache or built on the fly.\n- Backwards-compatible with existing non-Nix images.\n- Nix-snapshotter layers can be interleaved with normal layers.\n- Provides [CRI][cri] Image Service to allow Kubernetes to \"pull images\" from a\n  Nix store, allowing you to run containers without a Docker Registry.\n- Fully declarative Kubernetes resources, where the image reference is a Nix\n  store path \n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch2\u003eWhat is Nix?\u003c/h2\u003e\u003c/summary\u003e\n\nNix is a package manager / build system that has a complete understanding\nof build \u0026 runtime inputs for every package. Nix packages are stored in a\nglobal hashed path like: `/nix/store/s66mzxpvicwk07gjbjfw9izjfa797vsw-hello-2.12.1`.\nPackages usually follow a FHS convention, so Nix packages are typically\ndirectories containing other directories like bin, share, etc. For example, the\nhello binary would be available via `/nix/store/s66mzxpvicwk07gjbjfw9izjfa797vsw-hello-2.12.1/bin/hello`.\n\nRuntime dependencies down to glibc are also inside `/nix/store`, so it\nreally has a complete dependency graph. In the case of `hello`, the\ncomplete closure is following:\n\n```sh\n/nix/store/3n58xw4373jp0ljirf06d8077j15pc4j-glibc-2.37-8\n/nix/store/fz2c8qahxza5ygy4yvwdqzbck1bs3qag-libidn2-2.3.4\n/nix/store/q7hi3rvpfgc232qkdq2dacmvkmsrnldg-libunistring-1.1\n/nix/store/ryvnrp5n6kqv3fl20qy2xgcgdsza7i0m-xgcc-12.3.0-libgcc\n/nix/store/s66mzxpvicwk07gjbjfw9izjfa797vsw-hello-2.12.1\n```\n\nIf you inspect its ELF data, you can indeed see its linked against that\nspecific `glibc`:\n\n```sh\n$ readelf -d /nix/store/s66mzxpvicwk07gjbjfw9izjfa797vsw-hello-2.12.1/bin/hello | grep runpath\n 0x000000000000001d (RUNPATH)            Library runpath: [/nix/store/3n58xw4373jp0ljirf06d8077j15pc4j-glibc-2.37-8/lib]\n```\n\nThis means that a root filesystem containing that closure is sufficient to run\n`hello` even if it's dynamically linked. This is similar to minimal images\ncontaining a statically compiled go binary or like [distroless][distroless]\nwhich leverages [Bazel][bazel] to the same effect.\n\n\u003c/details\u003e\n\n## Getting started\n\n![Demo](docs/demo.gif)\n\nThe easiest way to try this out is run a NixOS VM with everything\npre-configured.\n\n\u003e [!NOTE]\n\u003e You'll need [Nix][nix] installed with [flake support][nix-flake] and [unified CLI][nix-command] enabled,\n\u003e which comes pre-enabled with [Determinate Nix Installer][nix-installer].\n\u003e\n\u003e \u003cdetails\u003e\n\u003e \u003csummary\u003eTrying without Nix installed\u003c/summary\u003e\n\u003e\n\u003e If you have [docker][docker] or another OCI runtime installed, you can run\n\u003e `docker run --rm -it nixpkgs/nix-flakes`:\n\u003e\n\u003e ```sh\n\u003e nix run github:pdtpartners/nix-snapshotter#vm\n\u003e ```\n\u003e \u003c/details\u003e\n\n```sh\nnix run \"github:pdtpartners/nix-snapshotter#vm\"\nnixos login: root # (Ctrl-a then x to quit)\nPassword: root\n\n# Running `pkgs.hello` image with nix-snapshotter\nnerdctl run ghcr.io/pdtpartners/hello\n\n# Running `pkgs.redis` image with kubernetes \u0026 nix-snapshotter\nkubectl apply -f /etc/kubernetes/redis/\n\n# Wait a few seconds... \nwatch kubectl get pods\n\n# And a kubernetes service will be ready to forward port 30000 to the redis\n# pod, so you can test it out with a `ping` command\nredis-cli -p 30000 ping\n```\n\nOr you can try running in rootless mode:\n\n```sh\nnix run \"github:pdtpartners/nix-snapshotter#vm-rootless\"\nnixos login: rootless # (Ctrl-a then x to quit)\nPassword: rootless\n\n# `nerdctl run` with rootless k3s containerd currently not supported yet\n# See: https://github.com/containerd/nerdctl/issues/2831\n#\n# If rootless kubernetes not needed, `nerdctl run` does work with rootless\n# containerd + nix-snapshotter.\n\n# Running `pkgs.redis` image with kubernetes \u0026 nix-snapshotter\nkubectl apply -f /etc/kubernetes/redis/\n\n# Wait a few seconds... \nwatch kubectl get pods\n\n# And a kubernetes service will be ready to forward port 30000 to the redis\n# pod, so you can test it out with a `ping` command\nredis-cli -p 30000 ping\n```\n\n## Installation\n\n[NixOS][nixos] and [Home Manager][home-manager] modules are provided for\neasy installation.\n\n\u003e [!IMPORTANT]\n\u003e Requires at least nixpkgs 23.05+\n\n- **Home Manager**\n\n  \u003cdetails\u003e\n  \u003csummary\u003eFlake\u003c/summary\u003e\n\n  ```nix\n  {\n    inputs = {\n      nixpkgs.url = \"github:NixOS/nixpkgs/nixos-unstable\";\n      home-manager = {\n        url = \"github:nix-community/home-manager\";\n        inputs.nixpkgs.follows = \"nixpkgs\";\n      };\n      nix-snapshotter = {\n        url = \"github:pdtpartners/nix-snapshotter\";\n        inputs.nixpkgs.follows = \"nixpkgs\";\n      };\n    };\n\n    outputs = { nixpkgs, home-manager, nix-snapshotter, ... }: {\n      homeConfigurations.myuser = home-manager.lib.homeManagerConfiguration {\n        pkgs = import nixpkgs { system = \"x86_64-linux\"; };\n        modules = [\n          {\n            home = {\n              username = \"myuser\";\n              homeDirectory = \"/home/myuser\";\n              stateVersion = \"23.11\";\n            };\n\n            programs.home-manager.enable = true;\n\n            # Let home-manager automatically start systemd user services.\n            # Will eventually become the new default.\n            systemd.user.startServices = \"sd-switch\";\n          }\n          ({ pkgs, ... }: {\n            # (1) Import home-manager module.\n            imports = [ nix-snapshotter.homeModules.default ];\n\n            # (2) Add overlay.\n            #\n            # NOTE: If using NixOS \u0026 home-manager.useGlobalPkgs = true, then add\n            # the overlay at the NixOS level.\n            nixpkgs.overlays = [ nix-snapshotter.overlays.default ];\n\n            # (3) Enable service.\n            virtualisation.containerd.rootless = {\n              enable = true;\n              nixSnapshotterIntegration = true;\n            };\n            services.nix-snapshotter.rootless = {\n              enable = true;\n            };\n\n            # (4) Add a containerd CLI like nerdctl.\n            home.packages = [ pkgs.nerdctl ];\n          })\n        ];\n      };\n    };\n  }\n  ```\n  \u003c/details\u003e\n\n  \u003cdetails\u003e\n  \u003csummary\u003eNon-flake\u003c/summary\u003e\n\n  ```nix\n  { pkgs, ... }:\n  let\n    nix-snapshotter = import (\n      builtins.fetchTarball \"https://github.com/pdtpartners/nix-snapshotter/archive/main.tar.gz\"\n    );\n\n  in {\n    imports = [\n      # (1) Import home-manager module.\n      nix-snapshotter.homeModules.default\n    ];\n\n    # (2) Add overlay.\n    #\n    # NOTE: If using NixOS \u0026 home-manager.useGlobalPkgs = true, then add\n    # the overlay at the NixOS level.\n    nixpkgs.overlays = [ nix-snapshotter.overlays.default ];\n\n    # (3) Enable service.\n    virtualisation.containerd.rootless = {\n      enable = true;\n      nixSnapshotterIntegration = true;\n    };\n    services.nix-snapshotter.rootless = {\n      enable = true;\n    };\n\n    # (4) Add a containerd CLI like nerdctl.\n    home.packages = [ pkgs.nerdctl ];\n  }\n  ```\n  \u003c/details\u003e\n\n- **NixOS**\n\n  \u003cdetails\u003e\n  \u003csummary\u003eFlake\u003c/summary\u003e\n\n  ```nix\n  {\n    inputs = {\n      nixpkgs.url = \"github:NixOS/nixpkgs/nixos-unstable\";\n      nix-snapshotter = {\n        url = \"github:pdtpartners/nix-snapshotter\";\n        inputs.nixpkgs.follows = \"nixpkgs\";\n      };\n    };\n\n    outputs = { nixpkgs, nix-snapshotter, ... }: {\n      nixosConfigurations.myhost = nixpkgs.lib.nixosSystem {\n        system = \"x86_64-linux\";\n        modules = [\n          ./hardware-configuration.nix\n          ({ pkgs, ... }: {\n            # (1) Import nixos module.\n            imports = [ nix-snapshotter.nixosModules.default ];\n\n            # (2) Add overlay.\n            nixpkgs.overlays = [ nix-snapshotter.overlays.default ];\n\n            # (3) Enable service.\n            virtualisation.containerd = {\n              enable = true;\n              nixSnapshotterIntegration = true;\n            };\n            services.nix-snapshotter = {\n              enable = true;\n            };\n\n            # (4) Add a containerd CLI like nerdctl.\n            environment.systemPackages = [ pkgs.nerdctl ];\n          })\n        ];\n      };\n    };\n  }\n  ```\n  \u003c/details\u003e\n\n  \u003cdetails\u003e\n  \u003csummary\u003eNon-flake\u003c/summary\u003e\n\n  ```nix\n  { pkgs, ... }:\n  let\n    nix-snapshotter = import (\n      builtins.fetchTarball \"https://github.com/pdtpartners/nix-snapshotter/archive/main.tar.gz\"\n    );\n\n  in {\n    imports = [\n      ./hardware-configuration.nix\n      # (1) Import nixos module.\n      nix-snapshotter.nixosModules.default\n    ];\n\n    # (2) Add overlay.\n    nixpkgs.overlays = [ nix-snapshotter.overlays.default ];\n\n    # (3) Enable service.\n    virtualisation.containerd = {\n      enable = true;\n      nixSnapshotterIntegration = true;\n    };\n    services.nix-snapshotter = {\n      enable = true;\n    };\n\n    # (4) Add a containerd CLI like nerdctl.\n    environment.systemPackages = [ pkgs.nerdctl ];\n  }\n  ```\n  \u003c/details\u003e\n\n- **Manual**\n\n  See the [manual installation docs][manual-install].\n\n## Usage\n\nSee [package.nix](package.nix) for the Nix interface. You can also repeat the\nasciinema demo above in\n[examples/declarative-k8s.nix](examples/declarative-k8s.nix).\n\n```nix\npkgs = import nixpkgs {\n  overlays = [ nix-snapshotter.overlays.default ];\n};\n\n# Builds a native Nix image but intended for an OCI-compliant registry.\nredis = pkgs.nix-snapshotter.buildImage {\n  name = \"ghcr.io/pdtpartners/redis\";\n  tag = \"latest\";\n  config.entrypoint = [ \"${pkgs.redis}/bin/redis-server\" ];\n};\n\n# Running \"${redis.copyToRegistry {}}/bin/copy-to-registry\" will copy it to\n# an OCI-compliant Registry. It will try to use your Docker credentials to push\n# if the target is DockerHub.\n\n# Builds a native Nix image with a special image reference. When running\n# the kubelet with `--image-service-endpoint` pointing to nix-snapshotter, then\n# it can resolve the image reference to this Nix package.\nredis' = pkgs.nix-snapshotter.buildImage {\n  name = \"redis\";\n  resolvedByNix = true;\n  config.entrypoint = [ \"${pkgs.redis}/bin/redis-server\" ];\n};\n\n# Fully declarative Kubernetes Pod, down to the image specification and its\n# contents.\nredisPod = pkgs.writeText \"redis-pod.json\" (builtins.toJSON {\n  apiVersion = \"v1\";\n  kind = \"Pod\";\n  metadata = {\n    name = \"redis\";\n    labels.name = \"redis\";\n  };\n  spec.containers = [{\n    name = \"redis\";\n    args = [ \"--protected-mode\" \"no\" ];\n    image = \"nix:0${redis'}\";\n    ports = [{\n      name = \"client\";\n      containerPort = 6379;\n    }];\n  }];\n});\n```\n\n\u003e [!NOTE]\n\u003e If you want to understand how `nix:0` gets resolved, take a look at the docs\n\u003e for [Image Service][image-service].\n\n## Contributing\n\nPull requests are welcome for any changes. Consider opening an issue to discuss\nlarger changes first to get feedback on the idea.\n\nPlease read [CONTRIBUTING](CONTRIBUTING.md) for development tips and\nmore details on contributing guidelines.\n\n## FAQ\n\n\u003e [!IMPORTANT]\n\u003e To understand how it works behind the scenes, see the\n\u003e [Architecture][architecture] docs for more details.\n\n1. What's the difference between this and [pkgs.dockerTools.buildImage][dockerTools]?\n\n\u003cdetails\u003e\n\u003csummary\u003eAnswer\u003c/summary\u003e\n\nThe upstream `buildImage` streams Nix packages into tarballs, compresses them\nand pushes them to an OCI registry. Since there is a limit to number of layers\nin an image, a heuristic is used to put popular packages together. There is\nlarge amount of duplication between your Nix binary cache and the Docker\nRegistry tarballs, and even between images that share packages as the layers may\nduplicate common packages due to the heuristic-based layering strategy.\n\nWith `pkgs.nix-snapshotter.buildImage`, containerd natively understand Nix\npackages, so everything is pulled at package granularity without the layer\nlimit. This means all the container content is either already in your Nix store\nor fetched from your Nix binary cache.\n\u003c/details\u003e\n\n2. When should I choose the rootful (normal) vs rootless mode?\n\n\u003cdetails\u003e\n\u003csummary\u003eAnswer\u003c/summary\u003e\n\nIf you are running a production server, it's best to use the rootful version\nas rootless containers is still in its early stages in the container ecosystem.\n\nHowever, if you are running it for personal use, do try the rootless variant\nfirst. Although less mature, it is the more secure mode as the container\nruntime runs as an unprivileged user. It can mitigate potential\ncontainer-breakout vulnerabilities, though its not a silver bullet.\n\nTypically, rootless mode is more complex to setup. But since it's already\ndistributed as a NixOS / Home Manager module, it's simple as enabling the\nservice.\n\nSee https://rootlesscontaine.rs for more details.\n\u003c/details\u003e\n\n3. What's the difference between this and [Nixery][nixery]?\n\n\u003cdetails\u003e\n\u003csummary\u003eAnswer\u003c/summary\u003e\n\nNixery exposes an API (in the form of an OCI registry) to dynamically build\nNix-based images. It has an [improved layering design][nixery-layers] compared\nto upstream `pkgs.dockerTools.buildImage` but is still fundamentally a\nheuristics- based layering strategy (see above), so it still suffers from the\nsame inefficiency in duplication. However, Nixery can totally start building\nnix-snapshotter images so we can have a Docker Registry that can dynamically\nbuild native Nix images. See this [Nixery issue][nixery-issue] to follow along\nthe progress.\n\n\u003c/details\u003e\n\n4. What's the difference between this and a nix-in-docker?\n\n\u003cdetails\u003e\n\u003csummary\u003eAnswer\u003c/summary\u003e\n\nIf you run nix inside a container (e.g. `nixos/nix` or `nixpkgs/nix-flake`)\nthen you are indeed fetching packages using the Nix store. However, each\ncontainer will have its own Nix store instead of de-duplicating at the host\nlevel.\n\nnix-snapshotter is intended to live on the host system (sibling to containerd\nand/or kubelet) so that multiple containers running different images can share\nthe underlying packages from the same Nix store.\n\n\u003c/details\u003e\n\n5. What's the difference between this and [nix2container][nix2container]?\n\n\u003cdetails\u003e\n\u003csummary\u003eAnswer\u003c/summary\u003e\n\nnix2container improves upon `pkgs.dockerTools.buildImage` in a few ways. First\nit does something similar to `pkgs.dockerTools.streamLayeredImage` where it\navoids writing Nix layer tarballs to Nix store and builds them JIT when\nexporting, like with it's passthru attribute `copyToRegistry`. This avoids\nwriting Nix layer tarballs into the Nix store unnecessarily.\n\nSecondly, it separates out image metadata and layer metadata. This means that\nwhen updating the image config, layers don't need to be rebuilt. Thirdly, each\nlayer metadata is in its own Nix package, so only updated layers need to be\nrebuilt.\n\nLastly, the layer metadata is a JSON that contains the Nix store paths along\nwith the digest which is computed from the layer tarball which is thrown away.\nThis lets the tool `skopeo` to only copy non-existing layers, which then builds\nthe requested layer tarballs again JIT.\n\nnix2container is a great improvement, but still suffers same problems pointed\nout in the `pkgs.dockerTools.buildImage` section. It duplicates data between\nNix binary cache and Docker Registry, and it duplicates packages between layers\ndue to using a similar heuristic-based strategy.\n\n`pkgs.nix-snapshotter.buildImage` has all the same improvements, except that\nwe do write the final image back to the Nix store since it's tiny and allows us\nto resolve image manifests via a Nix package.\n\n\u003c/details\u003e\n\n## License\n\nThe source code developed for nix-snapshotter is licensed under MIT License.\n\nThis project also contains modified portions of other projects that are\nlicensed under the terms of Apache License 2.0. See [NOTICE](NOTICE) for more\ndetails.\n\n[architecture]: docs/architecture.md\n[bazel]: https://bazel.build/\n[ci-badge]: https://github.com/pdtpartners/nix-snapshotter/actions/workflows/ci.yml/badge.svg\n[ci]: https://github.com/pdtpartners/nix-snapshotter/actions?query=workflow%3ACI\n[cri]: https://kubernetes.io/docs/concepts/architecture/cri/\n[containerd]: https://github.com/containerd/containerd\n[distroless]: https://github.com/GoogleContainerTools/distroless\n[docker]: https://www.docker.com/\n[dockerTools]: https://nixos.org/manual/nixpkgs/stable/#ssec-pkgs-dockerTools-buildImage\n[go-reference-badge]: https://pkg.go.dev/badge/github.com/pdtpartners/nix-snapshotter.svg\n[go-reference]: https://pkg.go.dev/github.com/pdtpartners/nix-snapshotter\n[go-report-card-badge]: https://goreportcard.com/badge/github.com/pdtpartners/nix-snapshotter\n[go-report-card]: https://goreportcard.com/report/github.com/pdtpartners/nix-snapshotter\n[home-manager]: https://github.com/nix-community/home-manager\n[image-service]: docs/architecture.md#image-service\n[manual-install]: docs/manual-install.md\n[nix2container]: https://github.com/nlewo/nix2container\n[nix-command]: https://zero-to-nix.com/concepts/nix#unified-cli\n[nixery]: https://nixery.dev/\n[nixery-issue]: https://github.com/tazjin/nixery/issues/160\n[nixery-layers]: https://tazj.in/blog/nixery-layers\n[nix-flake]: https://zero-to-nix.com/concepts/flakes\n[nix]: https://zero-to-nix.com/concepts/nix\n[nix-installer]: https://zero-to-nix.com/start/install\n[nixos]: https://zero-to-nix.com/concepts/nixos\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpdtpartners%2Fnix-snapshotter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpdtpartners%2Fnix-snapshotter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpdtpartners%2Fnix-snapshotter/lists"}