{"id":42752070,"url":"https://github.com/NixOS/nix-eval-jobs","last_synced_at":"2026-01-31T00:01:03.640Z","repository":{"id":36984586,"uuid":"347841046","full_name":"NixOS/nix-eval-jobs","owner":"NixOS","description":"Parallel nix evaluator with a streamable json output [maintainers @Mic92, @adisbladis]","archived":false,"fork":false,"pushed_at":"2026-01-28T19:35:49.000Z","size":693,"stargazers_count":217,"open_issues_count":16,"forks_count":43,"subscribers_count":8,"default_branch":"main","last_synced_at":"2026-01-29T10:14:06.280Z","etag":null,"topics":["build-with-buildbot"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NixOS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":"nix-community","open_collective":"nix-community"}},"created_at":"2021-03-15T04:53:42.000Z","updated_at":"2026-01-28T19:35:52.000Z","dependencies_parsed_at":"2023-12-25T02:22:17.021Z","dependency_job_id":"c11f531f-eb44-4163-9aeb-a5d43535bbba","html_url":"https://github.com/NixOS/nix-eval-jobs","commit_stats":{"total_commits":425,"total_committers":19,"mean_commits":22.36842105263158,"dds":0.6470588235294117,"last_synced_commit":"a3307ac4b38b6dae769f771c877a6f2c236ee238"},"previous_names":[],"tags_count":39,"template":false,"template_full_name":null,"purl":"pkg:github/NixOS/nix-eval-jobs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fnix-eval-jobs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fnix-eval-jobs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fnix-eval-jobs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fnix-eval-jobs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NixOS","download_url":"https://codeload.github.com/NixOS/nix-eval-jobs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fnix-eval-jobs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28923736,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-30T22:32:35.345Z","status":"ssl_error","status_checked_at":"2026-01-30T22:32:31.927Z","response_time":66,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["build-with-buildbot"],"created_at":"2026-01-29T20:00:29.882Z","updated_at":"2026-01-31T00:01:03.612Z","avatar_url":"https://github.com/NixOS.png","language":"C++","funding_links":["https://github.com/sponsors/nix-community","https://opencollective.com/nix-community"],"categories":["Command-Line Tools","C++"],"sub_categories":[],"readme":"# nix-eval-jobs\n\nThis project evaluates nix attribute sets in parallel with streamable json\noutput. This is useful for time and memory intensive evaluations such as NixOS\nmachines, i.e. in a CI context. The evaluation is done with a controllable\nnumber of threads that are restarted when their memory consumption exceeds a\ncertain threshold.\n\nTo facilitate integration, nix-eval-jobs creates garbage collection roots for\neach evaluated derivation (drv file, not the build) within the provided\nattribute. This prevents race conditions between the nix garbage collection\nservice and user-started nix builds processes.\n\n## Why using nix-eval-jobs?\n\n- Faster evaluation by using threads\n- Memory used for evaluation is reclaimed after nix-eval-jobs finish, so that\n  the build can use it.\n- Evaluation of jobs can fail individually\n\n## Example\n\nIn the following example we evaluate the hydraJobs attribute of the\n[patchelf](https://github.com/NixOS/patchelf) flake:\n\n```console\n$ nix-eval-jobs --gc-roots-dir gcroot --flake 'github:NixOS/patchelf#hydraJobs'\n{\"attr\":\"coverage\",\"attrPath\":[\"coverage\"],\"drvPath\":\"/nix/store/fmbqzaq8mim1423879lhn9whs6imx5w4-patchelf-coverage-0.18.0.drv\",\"inputDrvs\":{\"/nix/store/23632hx2c98lbbjld279dx0w08lxn6kp-hook.drv\":[\"out\"],\"/nix/store/6z1jfnqqgyqr221zgbpm30v91yfj3r45-bash-5.1-p16.drv\":[\"out\"],\"/nix/store/ap9g09fxbicj836zm88d56dn3ff4clxl-stdenv-linux.drv\":[\"out\"],\"/nix/store/c0gg7lj101xhd8v2b3cjl5dwwkpxfc0q-patchelf-tarball-0.18.0.drv\":[\"out\"],\"/nix/store/vslywm6kbazi37q1vbq8y7bi884yc6yx-lcov-1.16.drv\":[\"out\"],\"/nix/store/y964yq4vz1gsn7azd44vyg65gnr4gpvi-hook.drv\":[\"out\"]},\"name\":\"patchelf-coverage-0.18.0\",\"outputs\":{\"out\":\"/nix/store/gfni9sbhhwhxxfqziq1fs3n82bvw962l-patchelf-coverage-0.18.0\"},\"system\":\"x86_64-linux\"}\n{\"attr\":\"patchelf-win32\",\"attrPath\":[\"patchelf-win32\"],\"drvPath\":\"/nix/store/s38l0fg5ja6j8qpws7slw2ws0c6v0qcf-patchelf-i686-w64-mingw32-0.18.0.drv\",\"inputDrvs\":{\"/nix/store/6z1jfnqqgyqr221zgbpm30v91yfj3r45-bash-5.1-p16.drv\":[\"out\"],\"/nix/store/b2p151ilwqpd47fbmzz50a5cmj12ixbf-hook.drv\":[\"out\"],\"/nix/store/fbnhh18m4jh6cwa92am2sv3aqzjnzpdd-stdenv-linux.drv\":[\"out\"]},\"name\":\"patchelf-i686-w64-mingw32-0.18.0\",\"outputs\":{\"out\":\"/nix/store/w8r4h1xk71fryb99df8aszp83kfhw3bc-patchelf-i686-w64-mingw32-0.18.0\"},\"system\":\"x86_64-linux\"}\n{\"attr\":\"patchelf-win64\",\"attrPath\":[\"patchelf-win64\"],\"drvPath\":\"/nix/store/wxpym6d3dxr1w9syhinp7f058gwxfmd3-patchelf-x86_64-w64-mingw32-0.18.0.drv\",\"inputDrvs\":{\"/nix/store/6z1jfnqqgyqr221zgbpm30v91yfj3r45-bash-5.1-p16.drv\":[\"out\"],\"/nix/store/71lv5lsr1y59bv1b91jc9gg0n85kf1sq-stdenv-linux.drv\":[\"out\"],\"/nix/store/b2p151ilwqpd47fbmzz50a5cmj12ixbf-hook.drv\":[\"out\"]},\"name\":\"patchelf-x86_64-w64-mingw32-0.18.0\",\"outputs\":{\"out\":\"/nix/store/fkq5428l2xsb84yj0cc6q1lkvsrga7sv-patchelf-x86_64-w64-mingw32-0.18.0\"},\"system\":\"x86_64-linux\"}\n{\"attr\":\"release\",\"attrPath\":[\"release\"],\"drvPath\":\"/nix/store/3xpwg8f623dpkh6cblv2fzcq5n99xl0j-patchelf-0.18.0.drv\",\"inputDrvs\":{\"/nix/store/6z1jfnqqgyqr221zgbpm30v91yfj3r45-bash-5.1-p16.drv\":[\"out\"],\"/nix/store/9rmihrl9ys0sap6827xyns0y73vqafjx-patchelf-0.18.0.drv\":[\"out\"],\"/nix/store/am2zqx3pyc1i14f888jna785h0f841sg-patchelf-0.18.0.drv\":[\"out\"],\"/nix/store/c0gg7lj101xhd8v2b3cjl5dwwkpxfc0q-patchelf-tarball-0.18.0.drv\":[\"out\"],\"/nix/store/csjiccxbwpfv55m8kqs2xwrkkha14dnq-patchelf-0.18.0.drv\":[\"out\"],\"/nix/store/jsrnpxdx5vmpnakd9bkb3sk3lgh0k8hm-patchelf-0.18.0.drv\":[\"out\"],\"/nix/store/k8a51ax83554c67g98xf3y751vjgjs7m-patchelf-0.18.0.drv\":[\"out\"],\"/nix/store/wq3ncl207isqqkqmsa5ql4fg19jbrhxg-stdenv-linux.drv\":[\"out\"]},\"name\":\"patchelf-0.18.0\",\"outputs\":{\"out\":\"/nix/store/d0mzprvv3vhasj23r1a6qn8qip0srbc4-patchelf-0.18.0\"},\"system\":\"x86_64-linux\"}\n{\"attr\":\"tarball\",\"attrPath\":[\"tarball\"],\"drvPath\":\"/nix/store/c0gg7lj101xhd8v2b3cjl5dwwkpxfc0q-patchelf-tarball-0.18.0.drv\",\"inputDrvs\":{\"/nix/store/6z1jfnqqgyqr221zgbpm30v91yfj3r45-bash-5.1-p16.drv\":[\"out\"],\"/nix/store/9d754glmsvpjm5kxvgsjslvgv356kbmn-libtool-2.4.7.drv\":[\"out\"],\"/nix/store/ap9g09fxbicj836zm88d56dn3ff4clxl-stdenv-linux.drv\":[\"out\"],\"/nix/store/f1ksgsyplvb0sli4pls6k6vsfvmv519d-autoconf-2.71.drv\":[\"out\"],\"/nix/store/jf58lcnch1bmpbi2188c59w5zr1cqrx2-automake-1.16.5.drv\":[\"out\"]},\"name\":\"patchelf-tarball-0.18.0\",\"outputs\":{\"out\":\"/nix/store/72pz5awc7gpwdqxrdsy8j0bvg2n7z78q-patchelf-tarball-0.18.0\"},\"system\":\"x86_64-linux\"}\n```\n\nThe output here is newline-seperated json according to https://jsonlines.org.\n\nThe code is derived from [hydra's](https://github.com/nixos/hydra) eval-jobs\nexecutable.\n\n## Further options\n\n```console\nUSAGE: nix-eval-jobs [options] expr\n\n  --apply                Apply provided Nix function to each derivation. The result of this function will be serialized as a JSON value and stored inside `\"extraValue\"` key of the json line output.\n  --arg                  Pass the value *expr* as the argument *name* to Nix functions.\n  --arg-from-file        Pass the contents of file *path* as the argument *name* to Nix functions.\n  --arg-from-stdin       Pass the contents of stdin as the argument *name* to Nix functions.\n  --argstr               Pass the string *string* as the argument *name* to Nix functions.\n  --check-cache-status   Check if the derivations are present locally or in any configured substituters (i.e. binary cache). The information will be exposed in the `cacheStatus` field of the JSON output.\n  --constituents         whether to evaluate constituents for Hydra's aggregate feature\n  --debug                Set the logging verbosity level to 'debug'.\n  --eval-store\n            The [URL of the Nix store](@docroot@/store/types/index.md#store-url-format)\n            to use for evaluation, i.e. to store derivations (`.drv` files) and inputs referenced by them.\n\n  --expr                 treat the argument as a Nix expression\n  --flake                build a flake\n  --force-recurse        force recursion (don't respect recurseIntoAttrs)\n  --gc-roots-dir         garbage collector roots directory\n  --help                 show usage information\n  --impure               allow impure expressions\n  --include\n  Add *path* to search path entries used to resolve [lookup paths](@docroot@/language/constructs/lookup-path.md)\n\n  This option may be given multiple times.\n\n  Paths added through `-I` take precedence over the [`nix-path` configuration setting](@docroot@/command-ref/conf-file.md#conf-nix-path) and the [`NIX_PATH` environment variable](@docroot@/command-ref/env-common.md#env-NIX_PATH).\n\n  --log-format           Set the format of log output; one of `raw`, `internal-json`, `bar` or `bar-with-logs`.\n  --max-memory-size      maximum evaluation memory size in megabyte (4GiB per worker by default)\n  --meta                 include derivation meta field in output\n  --no-instantiate       don't instantiate (write) derivations, only evaluate (faster)\n  --option               Set the Nix configuration setting *name* to *value* (overriding `nix.conf`).\n  --override-flake       Override the flake registries, redirecting *original-ref* to *resolved-ref*.\n  --override-input       Override a specific flake input (e.g. `dwarffs/nixpkgs`).\n  --quiet                Decrease the logging verbosity level.\n  --reference-lock-file  Read the given lock file instead of `flake.lock` within the top-level flake.\n  --repair               During evaluation, rewrite missing or corrupted files in the Nix store. During building, rebuild missing or corrupted store paths.\n  --select               Apply provided Nix function to transform the evaluation root. This is applied before any attribute traversal begins. When used with --flake without a fragment, the function receives an attrset with 'outputs' and 'inputs'. When used with a flake fragment, it receives the selected attribute. Examples: --select 'flake: flake.outputs.packages' --select 'flake: flake.inputs.nixpkgs' --select 'outputs: outputs.packages.x86_64-linux'\n  --show-input-drvs      Show input derivations in the output for each derivation. This is useful to get direct dependencies of a derivation.\n  --show-trace           print out a stack trace in case of evaluation errors\n  --verbose              Increase the logging verbosity level.\n  --workers              number of evaluate workers\n```\n\n## Potential use-cases for the tool\n\n**Faster evaluator in deployment tools.** When evaluating NixOS machines,\nevaluation can take several minutes when run on a single core. This limits\nscalability for large deployments with deployment tools such as\n[NixOps](https://github.com/NixOS/nixops).\n\n**Faster evaluator in CIs.** In addition to evaluation speed for CIs, it is also\nuseful if evaluation of individual jobs in CIs can fail, as opposed to failing\nthe entire jobset. For CIs that allow dynamic build steps to be created, one can\nalso take advantage of the fact that nix-eval-jobs outputs the derivation path\nseparately. This allows separate logs and success status per job instead of a\nsingle large log file. In the\n[wiki](https://github.com/nix-community/nix-eval-jobs/wiki#ci-example-configurations)\nwe collect example ci configuration for various CIs.\n\n## Projects using nix-eval-jobs\n\n- [Hydra](https://github.com/NixOS/hydra) - The Nix-based continuous build\n  system\n- [nix-fast-build](https://github.com/Mic92/nix-fast-build) - Combine the power\n  of nix-eval-jobs with nix-output-monitor to speed-up your evaluation and\n  building process\n- [buildbot-nix](https://github.com/Mic92/buildbot-nix) - A nixos module to make\n  buildbot a proper Nix-CI\n- [colmena](https://github.com/zhaofengli/colmena) - A simple, stateless NixOS\n  deployment tool\n- [robotnix](https://github.com/danielfullmer/robotnix) - Build Android (AOSP)\n  using Nix, used in their\n  [CI](https://github.com/danielfullmer/robotnix/blob/38b80700ee4265c306dcfdcce45056e32ab2973f/.github/workflows/instantiate.yml#L18)\n\n## FAQ\n\n### How can I check if my package already have been uploaded in the binary cache?\n\nIf you provide the `--check-cache-status`, the json will contain a\n`\"cacheStatus\"` key in its json, with the following values:\n\n| Value    | Meaning                                                 |\n| -------- | ------------------------------------------------------- |\n| local    | Package is present locally                              |\n| cached   | Package is present in the binary cache, but not locally |\n| notBuilt | Package needs to be built.                              |\n\n### How can I evaluate nixpkgs?\n\nIf you want to evaluate nixpkgs in the same way\n[hydra](https://hydra.nixos.org/) does it, use this snippet:\n\n```console\n$ nix-eval-jobs --force-recurse pkgs/top-level/release.nix\n```\n\n### nix-eval-jobs consumes too much memory / is too slow\n\nBy default, nix-eval-jobs spawns one worker process and limits the memory usage\nfor each worker to 4GB.\n\nWhen using multiple workers, keep in mind that each worker process may need to\nre-evaluate shared dependencies of the attributes, which can introduce some\noverhead for each evaluation or cause workers to exceed their memory limit. You\ncan tune the following options:\n\n`--workers`: This option allows you to set the number of evaluation workers that\nnix-eval-jobs should spawn. You can increase or decrease this number to optimize\nthe evaluation speed and memory usage. For example, if you have a system with\nmany CPU cores but limited memory, you may want to reduce the number of workers\nto avoid exceeding the memory limit.\n\n`--max-memory-size`: This option allows you to adjust the memory limit for each\nworker process. By default, it's set to 4GiB, but you can increase or decrease\nthis value as needed. For example, if you have a system with a lot of memory and\nwant to speed up the evaluation, you may want to increase the memory limit to\nallow workers to cache more data in memory before getting restarted by\nnix-eval-jobs. Note that this is not a hard limit and memory usage may rise\nabove the limit momentarily before the worker process exits.\n\nOverall, tuning these options can help you optimize the performance and memory\nusage of nix-eval-jobs to better fit your system and evaluation needs.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNixOS%2Fnix-eval-jobs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FNixOS%2Fnix-eval-jobs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNixOS%2Fnix-eval-jobs/lists"}