{"id":13903246,"url":"https://github.com/nlewo/nix2container","last_synced_at":"2025-05-15T09:04:33.245Z","repository":{"id":37045016,"uuid":"441133093","full_name":"nlewo/nix2container","owner":"nlewo","description":"An archive-less dockerTools.buildImage implementation ","archived":false,"fork":false,"pushed_at":"2025-03-20T20:03:44.000Z","size":233,"stargazers_count":610,"open_issues_count":62,"forks_count":53,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-03-31T07:01:39.650Z","etag":null,"topics":["containers","docker","nix","nixos","oci","skopeo"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nlewo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-23T09:45:34.000Z","updated_at":"2025-03-28T13:34:08.000Z","dependencies_parsed_at":"2023-02-18T00:15:45.668Z","dependency_job_id":"ea96c77b-a5e1-44fb-90ae-39cf1b809336","html_url":"https://github.com/nlewo/nix2container","commit_stats":{"total_commits":132,"total_committers":20,"mean_commits":6.6,"dds":"0.38636363636363635","last_synced_commit":"fa6bb0a1159f55d071ba99331355955ae30b3401"},"previous_names":["nlewo/containers-image-nix"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nlewo%2Fnix2container","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nlewo%2Fnix2container/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nlewo%2Fnix2container/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nlewo%2Fnix2container/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nlewo","download_url":"https://codeload.github.com/nlewo/nix2container/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247615375,"owners_count":20967184,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["containers","docker","nix","nixos","oci","skopeo"],"created_at":"2024-08-06T22:01:54.513Z","updated_at":"2025-04-07T08:08:47.258Z","avatar_url":"https://github.com/nlewo.png","language":"Go","funding_links":[],"categories":["Building Images","Deployment Tools","Go","nix","docker","Development"],"sub_categories":["Builder","Discovery"],"readme":"# nix2container\n\nnix2container provides an efficient container development workflow\nwith images built by Nix: it doesn't write tarballs to the Nix store\nand allows to skip already pushed layers (without having to rebuild\nthem).\n\nThis is based on ideas developed in [this blog\npost](https://lewo.abesis.fr/posts/nix-build-container-image/).\n\n\n## Getting started\n\n```nix\n{\n  inputs.nix2container.url = \"github:nlewo/nix2container\";\n\n  outputs = { self, nixpkgs, nix2container }: let\n    pkgs = import nixpkgs { system = \"x86_64-linux\"; };\n    nix2containerPkgs = nix2container.packages.x86_64-linux;\n  in {\n    packages.x86_64-linux.hello = nix2containerPkgs.nix2container.buildImage {\n      name = \"hello\";\n      config = {\n        entrypoint = [\"${pkgs.hello}/bin/hello\"];\n      };\n    };\n  };\n}\n```\n\nThis image can then be loaded into Docker with\n\n```\n$ nix run .#hello.copyToDockerDaemon\n$ docker run hello:latest\nHello, world!\n```\n\n\n## More Examples\n\nTo load and run the `bash` example image into Podman:\n\n```\n$ nix run github:nlewo/nix2container#examples.bash.copyToPodman\n$ podman run -it bash\n```\n\n- [`bash`](./examples/bash.nix): Bash in `/bin/`\n- [`fromImage`](./examples/from-image.nix): Alpine as base image\n- [`fromImageManifest`](./examples/from-image-manifest.nix): Alpine as base image, from a stored `manifest.json`.\n- [`nginx`](./examples/nginx.nix)\n- [`nonReproducible`](./examples/non-reproducible.nix): with a non reproducible store path :/\n- [`openbar`](./examples/openbar.nix): set permissions on files (without root nor VM)\n- [`uwsgi`](./examples/uwsgi/default.nix): isolate dependencies in layers\n- [`layered`](./examples/layered.nix): build a layered image as described in [this blog post](https://grahamc.com/blog/nix-and-layered-docker-images)\n\n\n## Functions documentation\n\n### `nix2container.buildImage`\n\nFunction arguments are:\n\n- **`name`** (required): the name of the image.\n\n- **`tag`** (defaults to the image output hash): the tag of the image.\n\n- **`config`** (defaults to `{}`): an attribute set describing an image configuration as\n    defined in the [OCI image\n    specification](https://github.com/opencontainers/image-spec/blob/8b9d41f48198a7d6d0a5c1a12dc2d1f7f47fc97f/specs-go/v1/config.go#L23).\n\n- **`copyToRoot`** (defaults to `null`): a derivation (or list of\n    derivations) copied in the image root directory (store path\n    prefixes `/nix/store/hash-path` are removed, in order to relocate\n    them at the image `/`).\n\n    `pkgs.buildEnv` can be used to build a derivation which has to be copied to\n    the image root. For instance, to get bash and coreutils in the image `/bin`:\n    ```\n    copyToRoot = pkgs.buildEnv {\n      name = \"root\";\n      paths = [ pkgs.bashInteractive pkgs.coreutils ];\n      pathsToLink = [ \"/bin\" ];\n    };\n    ```\n\n- **`fromImage`** (defaults to `null`): an image that is used as base\n    image of this image; use `pullImage` or `pullImageFromManifest` to\n    supply this.\n\n- **`maxLayers`** (defaults to `1`): the maximum number of layers to\n    create. This is based on the store path \"popularity\" as described\n    in this [blog\n    post](https://grahamc.com/blog/nix-and-layered-docker-images). Note\n    this is applied on the image layers and not on layers added with\n    the `buildImage.layers` attribute.\n\n- **`perms`** (defaults to `[]`): a list of file permisssions which are\n    set when the tar layer is created: these permissions are not\n    written to the Nix store.\n\n    Each element of this permission list is a dict such as\n    ```\n    { path = \"a store path\";\n      regex = \".*\";\n      mode = \"0664\";\n    }\n    ```\n    The mode is applied on a specific path. In this path subtree,\n    the mode is then applied on all files matching the regex.\n\n- **`initializeNixDatabase`** (defaults to `false`): to initialize the\n    Nix database with all store paths added into the image. Note this\n    is only useful to run nix commands from the image, for instance to\n    build an image used by a CI to run Nix builds.\n\n- **`layers`** (defaults to `[]`): a list of layers built with the\n    buildLayer function: if a store path in deps or contents belongs\n    to one of these layers, this store path is skipped. This is pretty\n    useful to isolate store paths that are often updated from more\n    stable store paths, to speed up build and push time.\n\n\n### `nix2container.pullImage`\n\nPull an image from a container registry by name and tag/digest, storing the\nentirety of the image (manifest and layer tarballs) in a single store path.\nThe supplied `sha256` is the narhash of that store path.\n\nFunction arguments are:\n\n- **`imageName`** (required): the name of the image to pull.\n\n- **`imageDigest`** (required): the digest of the image to pull.\n\n- **`sha256`** (required): the sha256 of the resulting fixed output derivation.\n\n- **`os`** (defaults to `linux`)\n\n- **`arch`** (defaults to `x86_64`)\n\n- **`tlsVerify`** (defaults to `true`)\n\n\n### `nix2container.pullImageFromManifest`\n\nPull a base image from a container registry using a supplied manifest file, and the\nhashes contained within it. The advantages of this over the basic `pullImage`:\n\n- Each layer archive is in its own store path, which means each will download just once\n  and naturally deduplicate for multiple base images that share layers.\n- There is no Nix-specific hash, so it's possible update the base image by simply\n  re-fetching the `manifest.json` from the registry; no need to actually pull the whole\n  image just to compute a new narhash for it.\n\nWith this function the `manifest.json` acts as a lockfile meant to be stored in\nsource control alongside the Nix container definitions. As a convenience, the manifest\ncan be fetched/updated using the supplied passthru script, eg:\n\n```\nnix run .#examples.fromImageManifest.fromImage.getManifest \u003e examples/alpine-manifest.json\n```\n\nFunction arguments are:\n\n- **`imageName`** (required): the name of the image to pull.\n\n- **`imageManifest`** (required): the manifest file of the image to pull.\n\n- **`imageTag`** (defaults to `latest`)\n\n- **`os`** (defaults to `linux`)\n\n- **`arch`** (defaults to `x86_64`)\n\n- **`tlsVerify`** (defaults to `true`)\n\n- **`registryUrl`** (defaults to `registry.hub.docker.com`)\n\nNote that `imageTag`, `os`, and `arch` do not affect the pulled image; that is\ngoverned entirely by the supplied `manifest.json` file. These arguments are\nused for the manifest-selection logic in the included `getManifest` script.\n\n\n#### Authentication\n\nIf the Nix daemon is used for building, here is how to set up registry\nauthentication.\n\n1. `docker login URL` to whatever it is\n2. Copy `~/.docker/config.json` to `/etc/nix/skopeo/auth.json`\n3. Make the directory and all the files readable to the `nixbld` group:\n   ```\n   sudo chmod -R g+rx /etc/nix/skopeo\n   sudo chgrp -R nixbld /etc/nix/skopeo\n   ```\n4. Bind mount the file into the Nix build sandbox\n   ```\n   extra-sandbox-paths = /etc/skopeo/auth.json=/etc/nix/skopeo/auth.json\n   ```\n\nEvery time a new registry authentication has to be added, update\n`/etc/nix/skopeo/auth.json` file.\n\n\n### `nix2container.buildLayer`\n\nFor most use cases, this function is not required. However, it could be\nuseful to explicitly isolate some parts of the image in dedicated\nlayers, for caching (see the \"Isolate dependencies in dedicated\nlayers\" section) or non reproducibility (see the `reproducible`\nargument) purposes.\n\nFunction arguments are:\n\n- **`deps`** (defaults to `[]`): a list of store paths to include in the\n    layer.\n\n- **`copyToRoot`** (defaults to `null`): a derivation (or list of\n    derivations) copied in the image root directory (store path\n    prefixes `/nix/store/hash-path` are removed, in order to relocate\n    them at the image `/`).\n\n    `pkgs.buildEnv` can be used to build a derivation which has to be copied to\n    the image root. For instance, to get bash and coreutils in the image `/bin`:\n    ```\n    copyToRoot = pkgs.buildEnv {\n      name = \"root\";\n      paths = [ pkgs.bashInteractive pkgs.coreutils ];\n      pathsToLink = [ \"/bin\" ];\n    };\n    ```\n\n- **`reproducible`** (defaults to `true`): If `false`, the layer tarball\n    is stored in the store path. This is useful when the layer\n    dependencies are not bit reproducible: it allows to have the layer\n    tarball and its hash in the same store path.\n\n- **`maxLayers`** (defaults to `1`): the maximum number of layers to\n    create. This is based on the store path \"popularity\" as described\n    in this [blog\n    post](https://grahamc.com/blog/nix-and-layered-docker-images). Note\n    this is applied on the image layers and not on layers added with\n    the `buildLayer.layers` attribute.\n\n- **`perms`** (defaults to `[]`): a list of file permisssions which are\n    set when the tar layer is created: these permissions are not\n    written to the Nix store.\n\n    Each element of this permission list is a dict such as\n    ```\n    { path = \"a store path\";\n      regex = \".*\";\n      mode = \"0664\";\n    }\n    ```\n    The mode is applied on a specific path. In this path subtree,\n    the mode is then applied on all files matching the regex.\n\n- **`layers`** (defaults to `[]`): a list of layers built with the\n    `buildLayer` function: if a store path in deps or contents belongs\n    to one of these layers, this store path is skipped. This is pretty\n    useful to isolate store paths that are often updated from more\n    stable store paths, to speed up build and push time.\n\n- **`ignore`** (defaults to `null`): a store path to ignore when\n    building the layer. This is mainly useful to ignore the\n    configuration file from the container layer.\n\n- **`metadata`** (defaults to `{ created_by = \"nix2container\"; }`): an attribute\n    set containing this layer's `created_by`, `author` and `comment` values\n\n## Isolate dependencies in dedicated layers\n\nIt is possible to isolate application dependencies in a dedicated\nlayer. This layer is built by its own derivation: if storepaths\ncomposing this layer don't change, the layer is not rebuilt. Moreover,\nSkopeo can avoid to push this layer if it has already been pushed.\n\nLet's consider an `application` printing a conversation. This script\ndepends on `bash` and the `hello` binary. Because most of the changes\nconcern the script itself, it would be nice to isolate scripts\ndependencies in a dedicated layer: when we modify the script, we only\nneed to rebuild and push the layer containing the script. The layer\ncontaining dependencies won't be rebuilt and pushed.\n\nAs shown below, the `buildImage.layers` attribute allows to\nexplicitly specify a set of dependencies to isolate.\n\n```nix\n{ pkgs }:\nlet\n  application = pkgs.writeScript \"conversation\" ''\n    ${pkgs.hello}/bin/hello\n    echo \"Haaa aa... I'm dying!!!\"\n  '';\nin\npkgs.nix2container.buildImage {\n  name = \"hello\";\n  config = {\n    entrypoint = [\"${pkgs.bash}/bin/bash\" application];\n  };\n  layers = [\n    (pkgs.nix2container.buildLayer { deps = [pkgs.bash pkgs.hello]; })\n  ];\n}\n```\n\nThis image contains 2 layers: a layer with `bash` and `hello` closures\nand a second layer containing the script only.\n\nIn real life, the isolated layer can contains a Python environment or\nNode modules.\n\nSee\n[Nix \u0026 Docker: Layer explicitly without duplicate packages!](https://blog.eigenvalue.net/2023-nix2container-everything-once/)\nfor learning how to avoid duplicate store paths in your explicitly layered\nimages.\n\n## Quick and dirty benchmarks\n\nThe main goal of nix2container is to provide fast rebuild/push\ncontainer cycles. In the following, we provide an order of magnitude\nof rebuild and repush time, for the [`uwsgi` image](https://github.com/nlewo/nix2container/blob/c6a8d82f1cdd80fabb76e7c1459471e1ea95a080/examples/uwsgi/default.nix).\n\n**warning: this is quick and dirty benchmarks which only provide an order of magnitude**\n\nWe build the container and push the container. We then made a small\nchange in the `hello.py` file to trigger a rebuild and a push.\n\nMethod | Rebuild/repush time | Executed command\n---|---|---\nnix2container.buildImage | ~1.8s | `nix run .#example.uwsgi.copyToRegistry`\ndockerTools.streamLayeredImage | ~7.5s | `nix build .#example.uwsgi \\| docker load`\ndockerTools.buildImage | ~10s | `nix build .#example.uwsgi; skopeo copy docker-archive://./result docker://localhost:5000/uwsgi:latest`\n\nNote we could not compare the same distribution mechanisms because\n- Skopeo is not able to skip already loaded layers by the Docker daemon and\n- Skopeo failed to push to the registry an image streamed to stdin.\n\n\n## Run the tests\n\n```\nnix run .#tests.all\n```\n\nThis builds several example images with Nix, loads them with Skopeo,\nruns them with Podman, and test output logs.\n\nNot that, unfortunately, these tests are not executed in the Nix\nsandbox because it is currently not possible to run a container in the\nNix sandbox.\n\nIt is also possible to run a specific test:\n\n```\nnix run .#tests.basic\n```\n\n\n## The nix2container Go library\n\nThis library is currently used by the Skopeo `nix` transport available\nin [this branch](https://github.com/nlewo/image/tree/nix).\n\nFor more information, refer to [the Go\ndocumentation](https://pkg.go.dev/github.com/nlewo/nix2container).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnlewo%2Fnix2container","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnlewo%2Fnix2container","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnlewo%2Fnix2container/lists"}