{"id":30071014,"url":"https://github.com/NixOS/equinix-metal-builders","last_synced_at":"2025-08-08T12:08:16.688Z","repository":{"id":54688637,"uuid":"198527717","full_name":"NixOS/equinix-metal-builders","owner":"NixOS","description":"iPXE image for Nix builders on Equinix Metal's Spot market.","archived":true,"fork":false,"pushed_at":"2024-11-09T04:09:46.000Z","size":317,"stargazers_count":50,"open_issues_count":0,"forks_count":10,"subscribers_count":13,"default_branch":"main","last_synced_at":"2025-08-06T16:39:10.386Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Nix","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NixOS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"open_collective":"nixos"}},"created_at":"2019-07-24T00:25:14.000Z","updated_at":"2025-08-01T13:34:28.000Z","dependencies_parsed_at":"2024-02-26T03:26:37.214Z","dependency_job_id":"bf90b548-cffb-4fda-9da5-c8e37322c96a","html_url":"https://github.com/NixOS/equinix-metal-builders","commit_stats":{"total_commits":294,"total_committers":10,"mean_commits":29.4,"dds":0.13945578231292521,"last_synced_commit":"33f7ab6e3d79aa2dee4f525549efd5099da98ff5"},"previous_names":["grahamc/packet-nix-builder"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/NixOS/equinix-metal-builders","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fequinix-metal-builders","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fequinix-metal-builders/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fequinix-metal-builders/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fequinix-metal-builders/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NixOS","download_url":"https://codeload.github.com/NixOS/equinix-metal-builders/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fequinix-metal-builders/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269417542,"owners_count":24413384,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-08T02:00:09.200Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-08T12:07:57.545Z","updated_at":"2025-08-08T12:08:16.676Z","avatar_url":"https://github.com/NixOS.png","language":"Nix","funding_links":["https://opencollective.com/nixos"],"categories":["DevOps"],"sub_categories":[],"readme":"# Transient Nix Builders on Packet.com\n\nI use Packet.com's spot market to run transient, powerful Nix\nbuilders. The files here are custom to _my_ builders and _my_ use\ncase. However, they could easily serve as a nice template for you to\nuse.\n\nI would accept PRs parameterizing the code.\n\n# Principles of Operation\n\nThis repository creates a bootable iPXE image. We assume each boot\nstarts with an empty set of disks. In case this is not true, it erases\nall of the disks and creates one large ZFS stripe across all disks. If\nyou have any disks attached, it WILL erase it on every boot.\n\nEach machine is stateless. As soon as it boots it is ready to build.\nWhen the machine shuts down, all data is lost.\n\n# To customize\n\n1. Edit `./user.nix` to have your user and your user's key. If you\n   have a key which is only used by Nix's remote builder protocol,\n   then they might belong in in the sshKeys list at the top.\n\n2. Edit `./instances/m2.xlarge.x86.nix` to match the hardware you'll\n   be deploying to. These machines are all Packet.com's m2.xlarge.x86\n   type, so if you're also using those, it is ready to go.\n\n# Building\n\n## m2.xlarge.x86\n\nYou can simply `nix-build ./instances/m2.xlarge.x86.nix` in this\ndirectory and create a bootable image. On the other hand, I use\n`./build-x86_64-linux.sh`, which instantiates locally and builds on my\nnetboot server. The remote server builds much faster and saves my\nbattery life.\n\n## c2.large.arm\n\nIn principle this one is just as easy. If you're on a machine which\ncan build aarch64 binaries, then you can just run\n`nix-build ./instances/c2.large.arm.nix`. However, I am not and this\nis a bit annoying.\n\nTherefore, I've written `./build-aarch64-linux.sh` which requires a\nconfiguration file of this format:\n\n```\nbuildHost       root@my.aarch64.builder\npxeHost         netboot@my.netboot.server\npxeDir          /var/lib/nginx/netboot/\nopensslServer   my.netboot.server\nopensslPort     61616\n```\n\nThis will copy the derivations to `buildHost` for building, and then\nset up an openssl-wrapped netcat tunnel from `buildHost` to\n`opensslServer:opensslPort` for transfering the build products.\n\nMy laptop will SSH to the `pxeHost` and launch openssl and netcat,\nthen SSH to the `buildHost` and initiate a connection to\n`opensslServer:opensslPort`. If this doesn't work, make sure that\nport is open.\n\n# Deploying\n\nAfter building, copy the resulting directory's files to a web\naccessible directory and instruct the server to boot from the\nnetboot.ipxe file in the result.\n\nOn Packet, edit `./create-spot-request.sh` to include the Packet API\ninformation, and the URL of the netboot.ipxe. **This might be\nexpensive! Make sure you understand what it will cost.**\n\nI always use their spot market, but you could deploy this to a\nregular or reserved server just the same.\n\nIf you use their spot market, this repository leaves it up as an\nexercise to the reader to implement server discovery. Although, if\nyou're using Hydra, an importer exists at\nhttps://github.com/NixOS/nixos-org-configurations/tree/master/hydra-packet-importer\nalready.\n\n# Implementation Notes\n\n - A naive implementation of a remote Nix builder might stick with the\n   default unionfs. However, this approach uses a lot of extra CPU and\n   causes unstable and broken builds for more complex builds. Because\n   of this, we switched to making a full, proper filesystem across all\n   the disks present. See: https://github.com/NixOS/nixpkgs/issues/64126\n\n----\n\n_btw: I don't work for Packet. Just a fan._\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNixOS%2Fequinix-metal-builders","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FNixOS%2Fequinix-metal-builders","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNixOS%2Fequinix-metal-builders/lists"}