{"id":13687911,"url":"https://github.com/juspay/github-nix-ci","last_synced_at":"2025-08-09T04:26:09.853Z","repository":{"id":245340217,"uuid":"817962508","full_name":"juspay/github-nix-ci","owner":"juspay","description":"A simple NixOS \u0026 nix-darwin module for self-hosting GitHub runners","archived":false,"fork":false,"pushed_at":"2024-10-09T17:23:54.000Z","size":68,"stargazers_count":37,"open_issues_count":2,"forks_count":2,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-11-12T11:40:03.822Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Nix","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/juspay.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-20T20:14:48.000Z","updated_at":"2024-11-08T11:49:21.000Z","dependencies_parsed_at":"2024-09-12T04:21:58.741Z","dependency_job_id":"6ed73551-b3ff-45f2-921f-d327b5ee6d4e","html_url":"https://github.com/juspay/github-nix-ci","commit_stats":null,"previous_names":["juspay/github-ci-nix","juspay/github-nix-ci"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juspay%2Fgithub-nix-ci","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juspay%2Fgithub-nix-ci/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juspay%2Fgithub-nix-ci/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juspay%2Fgithub-nix-ci/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/juspay","download_url":"https://codeload.github.com/juspay/github-nix-ci/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251899235,"owners_count":21661973,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T15:01:03.221Z","updated_at":"2025-08-09T04:26:09.839Z","avatar_url":"https://github.com/juspay.png","language":"Nix","funding_links":[],"categories":["Nix"],"sub_categories":[],"readme":"[![project chat](https://img.shields.io/badge/zulip-join_chat-brightgreen.svg)](https://nixos.zulipchat.com/register/)\n\n# github-nix-ci\n\n`github-nix-ci` is a simple NixOS \u0026amp; nix-darwin module (wrapping[^wrap] the ones in nixpkgs and nix-darwin) for [self-hosting GitHub runners][gh-runner] on your machines (which could be a remote server or your personal macbook), so as to provide self-hosted CI for both personal and organization-wide repositories on GitHub.\n\n- [What it does](#what-it-does)\n- [Getting Started](#getting-started)\n  - [1. Create system configuration for the machine](#1-create-system-configuration-for-the-machine)\n  - [2. Create personal access tokens](#2-create-personal-access-tokens)\n  - [3. Configure `github-nix-ci` runners](#3-configure-github-nix-ci-runners)\n  - [4. Add the workflow to your repositories](#4-add-the-workflow-to-your-repositories)\n- [Production](#production)\n  - [Common issues](#common-issues)\n- [Examples](#examples)\n\n\n## What it does\n\nWe provide a [NixOS][nixos] and [nix-darwin] module[^wrap] that can be imported and utilized as easily as:\n\n```nix\n{\n  services.github-nix-ci = {\n    age.secretsDir = ./secrets;\n    personalRunners = {\n      \"srid/nixos-config\".num = 1;\n      \"srid/haskell-flake\".num = 3;\n    };\n    orgRunners = {\n      \"zed-industries\".num = 10;\n    };\n  };\n}\n```\n\nActivating this configuration spins up the required GitHub runners, with appropriate [labels][label] (hostname and Nix [system]s).\n\nIn conjunction with [nixci] (which is installed in the runners by default), your GitHub Actions workflow YAML can be as simple as follows in order to run CI, on your own machines, for your Nix flakes based projects:\n\n```yaml\njobs:\n  nix:\n    runs-on: ${{ matrix.system }}\n    strategy:\n      matrix:\n        system: [aarch64-darwin, x86_64-darwin, x86_64-linux]\n    steps:\n      - uses: actions/checkout@v4\n      - run: nixci build --systems \"github:nix-systems/${{ matrix.system }}\"\n```\n\n## Getting Started\n\nRepurposing an existing machine for running [self-hosted GitHub runners][gh-runner] involves the following steps.\n\n### 1. Create system configuration for the machine\n\n#### New configuration\n\nIf you do not already have a NixOS (for Linux) or nix-darwin (for macOS) system configuration, begin with [the templates](https://community.flake.parts/nixos-flake/templates) provided by `nixos-flake`. Alternatively, you may start from the minimal example ([`./example`](./example/flake.nix)) in this repo. If you use both the platforms, you can keep them in a single flake as the aforementioned example demonstrates.\n\n\u003e[!TIP]\n\u003e If you use `nixos-flake`, activating the configuration is as simple as running `nix run .#activate` (if done locally) or [`nix run .#deploy`](https://github.com/srid/nixos-flake/pull/54) if done remotely.\n\n\n#### Existing configuration\n\nIf you already have a NixOS or nix-darwin system configuration, you can use `github-nix-ci` as follows:\n\n1. Switch your configuration to [using flakes](https://nixos.asia/en/configuration-as-flake), if not already.[^non-flake]\n1. Add this repo as a flake input\n1. Add `inputs.github-nix-ci.nixosModules.default` (if NixOS) or `inputs.github-nix-ci.darwinModules.default` (if macOS/nix-darwin) to the `modules` list of your top-level system configuration.\n\n[^non-flake]: Non-flake users too can use this module by using [`fetchGit`](https://noogle.dev/f/builtins/fetchGit) or the like.\n\nTest that everything is okay by activating your configuration.\n\n### 2. Create personal access tokens\n\nFor our runners to be able to authorize against GitHub, we need to create **fine-grained personal access tokens** (PAC) for each user and organization. \n\n1. Go to https://github.com/settings/personal-access-tokens/new\n1. Create a fine-grained PAC\n    - Under **Resource owner**, choose the user or organization for whose repositories your runners will be building the CI for.\n    - Under **Repository access**, choose the appropriate option based on your needs\n    - Setup the necessary permissions\n        - If the token is for a personal account, under **Permissions -\u003e Repository permissions**, set *Administration* to \"Read and write\"\n        - If the token is for an organization, under **Permissions -\u003e Organization permissions**, set _Self-hosted runners_ to \"Read and write\"\n            - Don't forget to \"Allow public repositories\" under \"Actions -\u003e Runner groups -\u003e Default\" ([ref](https://stackoverflow.com/a/77415170)).\n\n#### Add tokens to your configuration using `agenix`\n\n\u003e[!TIP]\n\u003e Follow [the agenix tutorial](https://github.com/ryantm/agenix?tab=readme-ov-file#tutorial) for details. [This PR](https://github.com/srid/nixos-config/pull/57) in `srid/nixos-config` can also be used as reference. \n\n\u003e [!NOTE]\n\u003e This module does not *mandate* the use of `agenix`. If you use something else other than `agenix` for secrets management, set the `tokenFile` option manually.\n\n1. Create a `./secrets/secrets.nix` containing the SSH keys of yourself and the machines, as well as the list of token `.age` files (see next point). See [`./example/secrets/secrets.nix`](https://github.com/juspay/github-nix-ci/blob/main/example/secrets/secrets.nix) for reference.\n2. Create a `.age` file for each PAC secret you created in the previous section\n   - Run `agenix -e secrets/github-nix-ci/NAME.token.age` where `NAME` is the name of the github user or the organization the PAC is associated with, and then paste your token secret in it, saving the file. \n\n### 3. Configure `github-nix-ci` runners\n\nNow that you have set everything up, it is time to configure the runners themselves. For both NixOS and nix-darwin, you can add the following configuration:\n\n```nix\nservices.github-nix-ci = {\n  age.secretsDir = ./secrets; # Only if you use agenix\n  personalRunners = {\n    \"srid/emanote\".num = 1;\n    \"srid/haskell-flake\".num = 3;\n  };\n  orgRunners = {\n    \"zed-industries\".num = 10;\n  };\n};\n```\n\nThe above configuration adds 3 sets of GitHub runner daemons. Two of them are associated with the personal repos, whereas the 3rd set is associated with the organization (and thus *any* repository under that organization). The `num` property will spin-up that many runners for the associated repo or organization. Setting a `num` value that is greater than `1` enables you to run actions in parallel (upto the value of `num`).\n\nActivate your configuration, and visit **Settings -\u003e Actions -\u003e Runners** page of your repository or organization settings to confirm that the runners are ready and healthy.\n\n### 4. Add the workflow to your repositories\n\n\u003e[!WARNING]\n\u003e **A note on security** of self-hosted GitHub runners: GitHub [recommends](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#self-hosted-runner-security) using self-hosted runners only with *private* repositories, as forks *\"can potentially run dangerous code on [the] self-hosted runner machine by creating a pull request that executes the code in a workflow\"*. \n\u003e \n\u003e You can mitigate this risk by going to the **Fork pull request workflows from outside collaborators** setting (under **Settings -\u003e Actions -\u003e General**) and enabling \"Require approval for all outside collaborators\".\n\nFinally, you are equipped to add an [actions workflow file](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) to one of the repositories to test everything out. Here's an example if you have configured both NixOS and macOS runners:\n\n```yaml\n# ./.github/workflows/nix.yaml\nname: \"CI\"\non:\n  push:\n    branches:\n      - main\n  pull_request:\njobs:\n  nix:\n    runs-on: ${{ matrix.system }}\n    strategy:\n      matrix:\n        system: [aarch64-darwin, x86_64-linux]\n      fail-fast: false\n    steps:\n      - uses: actions/checkout@v4\n      - name: nixci\n        run: nixci --extra-access-tokens \"github.com=${{ secrets.GITHUB_TOKEN }}\" build --systems \"${{ matrix.system }}\"\n```\n\nThe above workflow uses [nixci] to build *all* outputs of your project flake.\n\n#### Matrix builds\n\nBecause [nixci] supports generating GitHub's [workflow matrix](https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs) configuration, you can use the following workflow YAML to schedule jobs at a fine-grained level to each runner:\n\n```yaml\n# ./.github/workflows/nix.yaml\nname: \"CI\"\non:\n  push:\n    branches:\n      - main\n  pull_request:\njobs:\n  \n  configure:\n    runs-on: x86_64-linux\n    outputs:\n      matrix: ${{ steps.set-matrix.outputs.matrix }}\n    steps:\n     - uses: actions/checkout@v4\n     - id: set-matrix\n       run: echo \"matrix=$(nixci gh-matrix --systems=x86_64-linux,aarch64-darwin | jq -c .)\" \u003e\u003e $GITHUB_OUTPUT\n  \n  nix:\n    runs-on: ${{ matrix.system }}\n    permissions:\n      contents: read\n    needs: configure\n    strategy:\n      matrix: ${{ fromJson(needs.configure.outputs.matrix) }}\n      fail-fast: false\n    steps:\n      - uses: actions/checkout@v4\n      - run: |\n          nixci \\\n            --extra-access-tokens \"github.com=${{ secrets.GITHUB_TOKEN }}\" \\\n            build \\\n            --systems \"${{ matrix.system }}\" \\\n            .#default.${{ matrix.subflake}}\n```\n\nSee [srid/haskell-flake](https://github.com/srid/haskell-flake/blob/master/.github/workflows/ci.yaml) for a  real-world example.\n\n## Production\n\n### Common issues\n\n#### `Forbidden Runner version ... is deprecated and cannot receive messages.`\n\nYour runner may suddenly crash with an error like this:\n\n```\nJun 27 22:39:54 dosa Runner.Listener[424134]: An error occured: Error: Forbidden Runner version v2.316.1 is deprecated and cannot receive messages.\n```\n\nTo resolve this, you need to update your github runner package by updating the `nixpkgs` flake input and then re-deploy. See https://github.com/actions/runner/issues/3332#issuecomment-2187929070\n\n\u003e[!TIP]\n\u003e\n\u003e The `github-runner` package is auto-updated in nixpkgs by the r-ryantm bot ([example](https://github.com/NixOS/nixpkgs/pull/316806)), and then automatically gets backported ([example](https://github.com/NixOS/nixpkgs/pull/316888)) to stable NixOS releases.\n\n\n## Examples\n\n- [`./example`](./example)\n- [`srid/nixos-config`](https://github.com/srid/nixos-config/pull/60)\n\n\n\n[nixci]: https://github.com/srid/nixci\n[nix-darwin]: https://nixos.asia/en/nix-darwin\n[nixos]: https://nixos.asia/en/nixos\n[label]: https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/using-labels-with-self-hosted-runners\n[system]: https://flake.parts/system\n[gh-runner]: https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners\n[pac]: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-fine-grained-personal-access-token\n[ragenix]: https://github.com/yaxitech/ragenix\n[flake-parts]: https://nixos.asia/en/flake-parts\n\n[^wrap]: Our module wraps the upstream [NixOS](https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/tools/continuous-integration/github-runner) and [nix-darwin](https://github.com/LnL7/nix-darwin/tree/master/modules/services/github-runner) modules, whilst providing a platform-independent module interface, in addition to wiring up anything else required (users, secrets) to get going easily.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuspay%2Fgithub-nix-ci","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjuspay%2Fgithub-nix-ci","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuspay%2Fgithub-nix-ci/lists"}