{"id":13440485,"url":"https://github.com/Mic92/nix-fast-build","last_synced_at":"2025-03-20T09:33:23.531Z","repository":{"id":193465695,"uuid":"688842677","full_name":"Mic92/nix-fast-build","owner":"Mic92","description":"Combine the power of nix-eval-jobs with nix-output-monitor to speed-up your evaluation and building process.","archived":false,"fork":false,"pushed_at":"2024-10-09T07:36:09.000Z","size":104,"stargazers_count":231,"open_issues_count":7,"forks_count":10,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-11T23:34:43.753Z","etag":null,"topics":["build-with-buildbot","managed-by-renovate"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mic92.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-08T08:11:43.000Z","updated_at":"2024-10-09T19:13:11.000Z","dependencies_parsed_at":"2023-12-03T12:35:19.278Z","dependency_job_id":"b0d5171f-b15d-413f-a1f5-acefdfcb5528","html_url":"https://github.com/Mic92/nix-fast-build","commit_stats":null,"previous_names":["mic92/nix-ci-build","mic92/nix-fast-build"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mic92%2Fnix-fast-build","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mic92%2Fnix-fast-build/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mic92%2Fnix-fast-build/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mic92%2Fnix-fast-build/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mic92","download_url":"https://codeload.github.com/Mic92/nix-fast-build/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244586077,"owners_count":20476876,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["build-with-buildbot","managed-by-renovate"],"created_at":"2024-07-31T03:01:23.190Z","updated_at":"2025-03-20T09:33:23.526Z","avatar_url":"https://github.com/Mic92.png","language":"Python","funding_links":[],"categories":["HarmonyOS","Python","Development"],"sub_categories":["Windows Manager"],"readme":"# nix-fast-build 🚀 (previously known as nix-ci-build)\n\nCombine the power of `nix-eval-jobs` with `nix-output-monitor` to speed-up your\nevaluation and building process. `nix-fast-build` an also integrates with remote\nmachines by uploading the current flake, performing the evaluation/build\nremotely, and then transferring the resultant store paths back to you.\n\n## Why `nix-fast-build`?\n\n**Problem**: Evaluating and building big flakes i.e. with numerous NixOS\nmachines can be painfully slow. For instance, rebuilding the already-compiled\n[disko integration test suite](https://github.com/nix-community/disko) demands\n1:50 minutes on an AMD Ryzen 9 7950X3D. But, it only takes a\n[10 seconds](https://github.com/Mic92/nix-fast-build/issues/1) with\n`nix-fast-build`.\n\n**Solution**: `nix-fast-build` makes builds faster by evaluating and building\nyour nix packages concurrently, reducing the overall time.\n\n## How Does It Work?\n\nUnder the hood:\n\n1. It leverages the output from `nix-eval-jobs` to evaluate flake attributes in\n   parallel.\n2. As soon as attributes complete evaluation, `nix-fast-build` initiates their\n   build, even if the overall evaluation is ongoing.\n3. Lastly, `nix-output-monitor` to show the build progress nicely.\n4. (Optional) Once a build finishes, `nix-fast-build` can initiate its upload to\n   a designated remote binary cache.\n\n## Usage\n\nTo get started, run:\n\n```console\n$ nix-fast-build\n```\n\nor:\n\n```\n$ nix run github:Mic92/nix-fast-build\n```\n\nThis command will concurrently evaluate all systems in `.#checks` and build the\nattributes in `.#checks.$currentSystem`.\n\n---\n\nEnjoy faster and more efficient NixOS builds with `nix-fast-build`!\n\n## Remote building\n\nWhen leveraging the remote-builder protocol, uploading pre-built paths or\nsources from the local machine can often turn into a bottleneck.\n`nix-fast-build` does not use the remote-builder protocol. Instead it uploads\nonly the flake and executes all evaluation/build operations on the remote end.\nAt the end `nix-fast-build` will download the finished builds to the local\nmachine while not having to download all build dependencies in between.\n\nHere is how to use it:\n\n```\nnix run github:Mic92/nix-fast-build -- --remote youruser@yoursshhostname\n```\n\nReplace `youruser@yoursshhostname` with your SSH login credentials for the\ntarget machine. Please note that as of now, you must be recognized as a trusted\nuser on the remote endpoint to access this feature.\n\n## CI-Friendly Output\n\nBy default, `Nix-output-monitor` (abbreviated as `nom`) updates its output every\n0.5 seconds. In standard terminal environments, this frequent update is\nunnoticeable, as `nom` erases the previous output before displaying the new one.\nHowever, in Continuous Integration (CI) systems, each update appears as a\nseparate line of output.\n\nTo make output more concise for CI environments, use the `--no-nom` flag. This\nreplaces `nom` with a streamlined status reporter, which updates only when\nthere's a change in the number of pending builds, uploads, or downloads.\n\n## Avoiding Redundant Package Downloads\n\nBy default, `nix build` will download pre-built packages, leading to needless\ndownloads even when there are no changes to any package. This can be especially\nburdensome for CI environments without a persistent Nix store, such as GitHub\nActions.\n\nTo optimize this, use the `--skip-cached` flag with `nix-fast-build`. This\nensures that only those packages missing from the binary caches will be built.\n\n## Specifying Build Systems\n\nBy default, `nix-fast-build` evaluates all architectures but only initiates\nbuilds for the current system. You can modify this behavior with the `--systems`\nflag. For instance, using `--systems \"aarch64-linux x86_64-linux\"` will prompt\nbuilds for both `aarch64-linux` and `x86_64-linux` architectures. Ensure that\nyour system is capable of building for the specified architectures, either\nlocally or through the remote builder protocol.\n\n## Building different flake attributes\n\n`nix-fast-build` by default builds `.#checks.$currentSystem`, which refers to\nall checks for the current flake. You can modify this default behavior by using\nthe `--flake` flag to specify a different attribute path.\n\nExample:\n\n```console\n$ nix run github:Mic92/nix-fast-build -- --flake github:NixOS/nixpkgs#legacyPackages.x86_64-linux.hello\n```\n\n**Note:** Always provide the complete flake path. Unlike `nix build`,\n`nix-fast-build` does not iterate over different attributes; the full path must\nbe explicitly stated.\n\n## Only evaluate the current system\n\nBy default nix-fast-build will evaluate all systems in `.#checks`, you can limit\nit to the current system by using this command:\n\n```console\n$ nix run github:Mic92/nix-fast-build -- --skip-cached --no-nom --flake \".#checks.$(nix eval --raw --impure --expr builtins.currentSystem)\"\n```\n\n## Cachix support\n\nnix-fast-build can upload to cachix like this:\n\n```console\n$ nix-fast-build --cachix-cache mic92\n```\n\nnix-fast-build assumes that your current machine is either logged in to cachix\nor has the environment variables `CACHIX_SIGNING_KEY` or `CACHIX_AUTH_TOKEN`\nset. These environment variables are currently not propagated to ssh when using\nthe `--remote` flag, instead the user is expected that cachix credentials are\nconfigured on the remote machine.\n\n## Attic support\n\nnix-fast-build can upload to attic like this:\n\n```console\n$ nix-fast-build --attic-cache mic92\n```\n\nnix-fast-build assumes that your current machine is either logged in to attic.\nAuthentication is not propagated to ssh when using the `--remote` flag, instead\nthe user is expected that attic credentials are configured on the remote\nmachine.\n\n## Machine-readable builds results\n\nnix-fast-build supports both its own json format and junit:\n\nExample for json output:\n\n```console\nnix-fast-build --result-file result.json\ncat ./result.json\n{\n   \"results\": [\n     {\n       \"attr\": \"riscv64-linux.package-default\",\n       \"duration\": 0.0,\n       \"error\": null,\n       \"success\": true,\n       \"type\": \"EVAL\"\n     },\n# ...\n```\n\nExample for junit result output:\n\n```console\nnix-fast-build --result-format junit --result-file result.xml\n```\n\n```console\nnix-shell -p python3Packages.junit2html --run 'junit2html result.xml result.html'\n```\n\n## Reference\n\n```console\nusage: nix-fast-build [-h] [-f FLAKE] [-j MAX_JOBS] [--option name value]\n                      [--remote-ssh-option name value]\n                      [--cachix-cache CACHIX_CACHE] \n                      [--attic-cache ATTIC_CACHE] [--no-nom]\n                      [--systems SYSTEMS] [--retries RETRIES] [--no-link]\n                      [--out-link OUT_LINK] [--remote REMOTE]\n                      [--always-upload-source] [--no-download] [--skip-cached]\n                      [--copy-to COPY_TO] [--debug]\n                      [--eval-max-memory-size EVAL_MAX_MEMORY_SIZE]\n                      [--eval-workers EVAL_WORKERS]\n                      [--result-file RESULT_FILE]\n                      [--result-format {json,junit}]\n\noptions:\n  -h, --help            show this help message and exit\n  -f FLAKE, --flake FLAKE\n                        Flake url to evaluate/build (default: .#checks\n  -j MAX_JOBS, --max-jobs MAX_JOBS\n                        Maximum number of build jobs to run in parallel (0 for\n                        unlimited)\n  --option name value   Nix option to set\n  --remote-ssh-option name value\n                        ssh option when accessing remote\n  --cachix-cache CACHIX_CACHE\n                        Cachix cache to upload to\n  --attic-cache ATTIC_CACHE\n                        Attic cache to upload to\n  --no-nom              Don't use nix-output-monitor to print build output\n                        (default: false)\n  --systems SYSTEMS     Space-separated list of systems to build for (default:\n                        current system)\n  --retries RETRIES     Number of times to retry failed builds\n  --no-link             Do not create an out-link for builds (default: false)\n  --out-link OUT_LINK   Name of the out-link for builds (default: result)\n  --remote REMOTE       Remote machine to build on\n  --always-upload-source\n                        Always upload sources to remote machine. This is\n                        needed if the remote machine cannot access all sources\n                        (default: false)\n  --no-download         Do not download build results from remote machine\n  --skip-cached         Skip builds that are already present in the binary\n                        cache (default: false)\n  --copy-to COPY_TO     Copy build results to the given path (passed to nix\n                        copy, i.e. file:///tmp/cache?compression=none)\n  --debug               debug logging output\n  --eval-max-memory-size EVAL_MAX_MEMORY_SIZE\n                        Maximum memory size for nix-eval-jobs (in MiB) per\n                        worker. After the limit is reached, the worker is\n                        restarted.\n  --eval-workers EVAL_WORKERS\n                        Number of evaluation threads spawned\n  --result-file RESULT_FILE\n                        File to write build results to\n  --result-format {json,junit}\n                        Format of the build result file\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMic92%2Fnix-fast-build","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FMic92%2Fnix-fast-build","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMic92%2Fnix-fast-build/lists"}