{"id":15171076,"url":"https://github.com/nixos/hydra-provisioner","last_synced_at":"2025-10-01T06:32:02.095Z","repository":{"id":66132983,"uuid":"41431817","full_name":"NixOS/hydra-provisioner","owner":"NixOS","description":"On-demand provisioning tool for Hydra","archived":true,"fork":false,"pushed_at":"2019-04-25T09:19:53.000Z","size":15,"stargazers_count":29,"open_issues_count":1,"forks_count":11,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-10-30T08:26:13.659Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NixOS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null},"funding":{"open_collective":"nixos"}},"created_at":"2015-08-26T14:55:28.000Z","updated_at":"2024-02-27T09:34:49.000Z","dependencies_parsed_at":"2023-07-02T09:30:21.356Z","dependency_job_id":null,"html_url":"https://github.com/NixOS/hydra-provisioner","commit_stats":{"total_commits":25,"total_committers":4,"mean_commits":6.25,"dds":"0.16000000000000003","last_synced_commit":"ecbfc27d754d28d5393e22915d3322c7dd116ad8"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fhydra-provisioner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fhydra-provisioner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fhydra-provisioner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NixOS%2Fhydra-provisioner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NixOS","download_url":"https://codeload.github.com/NixOS/hydra-provisioner/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234837067,"owners_count":18894529,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-27T09:00:22.985Z","updated_at":"2025-10-01T06:31:56.804Z","avatar_url":"https://github.com/NixOS.png","language":"Python","funding_links":["https://opencollective.com/nixos"],"categories":[],"sub_categories":[],"readme":"# Hydra provisioner\n\n`hydra-provisioner` is a script that automatically creates and destroys build machines (typically EC2 instances) for a Hydra server based on the state of the Hydra queue runner. For instance, if there are many runnable build steps for a particular system type (e.g. `x86_64-linux`), it will create additional build machines, then destroy them when they are no longer needed. The machines are managed using NixOps.\n\nTo run this script:\n```bash\n$ hydra-provisioner conf.nix\n```\nThis command should be run periodically (e.g. every 5 minutes). `conf.nix` is a Nix expression containing a specification of when and how to create machines. For example:\n```nix\n{\n\n  # Tag used for NixOps deployments created by the provisioner. Useful\n  # if you're running multiple provisioners.\n  #tag = \"hydra-provisioned\";\n\n  # The spec must contain one or more sets named systemTypes.\u003ctype\u003e,\n  # where \u003ctype\u003e is a Nix system type such as \"x86_64-linux\". You \n  # can also list system features (e.g. \"x86_64-linux:benchmark\"), in \n  # which case only build steps that have \"requiredSystemFeatures\" set to\n  # the listed features will be executed on the machines created here.\n  systemTypes.x86_64-linux = {\n\n    # Path to NixOps module defining the deployment for this type.\n    nixopsExpr = builtins.toPath ./deployment.nix;\n\n    # The minimum number of machines to keep around for this type.\n    #minMachines = 0;\n\n    # The maximum number of machines to provision for this type.\n    #maxMachines = 1;\n\n    # Value subtracted from the number of runnables of this type. This\n    # is the number of runnables to be performed by non-provisioned\n    # machines, before the provisioner kicks in to create more\n    # machines.\n    #ignoredRunnables = 0;\n\n    # How many machines should be created given the number of\n    # runnables. For instance, if there are 10 runnables and\n    # runnablesPerMachine is 5, then 2 machines will be created.\n    #runnablesPerMachine = 10;\n\n    # How many jobs can be run concurrently on machines of this type.\n    #maxJobs = 1;\n\n    # The speed factor.\n    #speedFactor = 1;\n\n    # The path of the SSH private key.\n    #sshKey = \"/var/lib/hydra/queue-runner/.ssh/id_buildfarm\";\n\n    # Whether to stop or destroy the machine when it's idle.\n    #stopOnIdle = false;\n\n    # Grace period in seconds before an idle machine is stopped or\n    # destroyed. Thus, if Hydra load increases in the meantime, the\n    # machine can be put back in action. Note that regardless of this \n    # setting, EC2 instances are not stopped or destroyed until their \n    # current hour of execution time has nearly expired.\n    #gracePeriod = 0;\n\n  };\n  \n  # Command for getting the Hydra queue status. Useful if the provisioner \n  # runs on a different machine from the queue runner.\n  #sshCommand = [\"hydra-queue-runner\", \"--status\"]\n  \n  # Command for writing the queue runner's machines file. The contents are \n  # passed via stdin.\n  #updateCommand = [ \"/bin/sh\" \"-c\" \"cat \u003e /var/lib/hydra/provisioner/machines\" ];\n}\n```\nThe NixOps specification (e.g. `deployment.nix`) must declare one machine named `machine`. The provisioner will create a separate NixOps deployment for each machine that it creates. A typical NixOps specification looks like this:\n```nix\n# The \"type\" argument corresponds to the system type (such as \n# \"x86_64-linux:benchmark\"), and can be used for creating different\n# kinds of machines from the same NixOps specification.\n{ type, tag, ... }:\n\n{\n\n  machine =\n    { config, lib, pkgs, ... }:\n    {\n      deployment.targetEnv = \"virtualbox\";\n      \n      # The queue runner will perform build actions via \"nix-store --serve\" \n      # on root@\u003cmachine\u003e, so this machine needs an authorized key for that.\n      users.extraUsers.root.openssh.authorizedKeys.keys = lib.singleton ''\n        command=\"nix-store --serve --write\" ssh-dss AAAAB3NzaC1...\n      '';\n      \n      # Currently, Hydra works better with the Nix 1.10 prerelease.\n      nix.package = pkgs.nixUnstable;\n      \n      # Frequent garbage collection is a good idea for build machines.\n      nix.gc.automatic = true;\n      nix.gc.dates = \"*:0/30\";\n    };\n\n}\n```\n\nA real-world configuration (for `hydra.nixos.org`) can be found at https://github.com/NixOS/nixos-org-configurations/tree/master/hydra-provisioner.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnixos%2Fhydra-provisioner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnixos%2Fhydra-provisioner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnixos%2Fhydra-provisioner/lists"}