{"id":47726600,"url":"https://github.com/tommeier/pipette-buildkite-plugin","last_synced_at":"2026-04-06T00:01:28.890Z","repository":{"id":347902233,"uuid":"1195714063","full_name":"tommeier/pipette-buildkite-plugin","owner":"tommeier","description":"Declarative Buildkite pipeline generation for monorepos, written in Elixir","archived":false,"fork":false,"pushed_at":"2026-04-01T04:07:46.000Z","size":234,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-03T06:29:01.713Z","etag":null,"topics":["buildkite-plugin","ci","elixir","monorepo","pipeline"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tommeier.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-30T01:46:41.000Z","updated_at":"2026-04-01T04:07:47.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/tommeier/pipette-buildkite-plugin","commit_stats":null,"previous_names":["tommeier/pipette-buildkite-plugin"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/tommeier/pipette-buildkite-plugin","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tommeier%2Fpipette-buildkite-plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tommeier%2Fpipette-buildkite-plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tommeier%2Fpipette-buildkite-plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tommeier%2Fpipette-buildkite-plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tommeier","download_url":"https://codeload.github.com/tommeier/pipette-buildkite-plugin/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tommeier%2Fpipette-buildkite-plugin/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31379453,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-03T21:40:47.592Z","status":"ssl_error","status_checked_at":"2026-04-03T21:40:05.436Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["buildkite-plugin","ci","elixir","monorepo","pipeline"],"created_at":"2026-04-02T20:36:28.999Z","updated_at":"2026-04-03T22:01:52.392Z","avatar_url":"https://github.com/tommeier.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pipette\n\n[![Hex.pm](https://img.shields.io/hexpm/v/buildkite_pipette.svg)](https://hex.pm/packages/buildkite_pipette)\n[![CI](https://github.com/tommeier/pipette-buildkite-plugin/actions/workflows/ci.yml/badge.svg)](https://github.com/tommeier/pipette-buildkite-plugin/actions)\n[![License](https://img.shields.io/hexpm/l/buildkite_pipette.svg)](LICENSE)\n\n**Declarative Buildkite pipeline generation for monorepos, written in Elixir.**\n\nDefine your CI pipeline with a declarative DSL powered by [Spark](https://hexdocs.pm/spark) — scope-based change detection, branch policies, commit message targeting, dependency graphs, and dynamic group generation. Compile-time validation catches misconfigured scopes, missing dependencies, and label conflicts before your pipeline runs.\n\n## Features\n\n- **Scope-based activation** — map file globs to named scopes; only groups whose scope matches changed files will run\n- **Branch policies** — run all groups on `main`, restrict to specific scopes on release branches, use file-based detection elsewhere\n- **Commit message targeting** — `[ci:api]` or `[ci:api/test]` in commit messages to run specific groups/steps\n- **Dependency propagation** — groups that `depends_on` an active group are pulled in automatically; scopeless groups activate when any dependency is active\n- **Force activation** — environment variables like `FORCE_DEPLOY=true` bypass scope detection to activate specific groups\n- **Dynamic groups** — `extra_groups` callback to generate groups at runtime (e.g. discovering packages in a directory)\n- **Branch-scoped groups** — `only: \"main\"` restricts groups to specific branches\n- **Trigger steps** — fire downstream Buildkite pipelines when conditions are met\n- **Compile-time validation** — Spark verifiers catch scope ref errors, dependency cycles, and label collisions at compile time\n- **YAML output** — generates valid Buildkite pipeline YAML via `ymlr`\n\n## Quick Start\n\nDefine a pipeline module:\n\n```elixir\ndefmodule MyApp.Pipeline do\n  use Pipette.DSL\n\n  branch(\"main\", scopes: :all, disable: [:targeting])\n\n  scope(:api_code, files: [\"apps/api/**\", \"mix.exs\"])\n  scope(:web_code, files: [\"apps/web/**\", \"package.json\"])\n  scope(:infra_code, files: [\"infra/**\"], exclude: [\"**/*.md\"])\n\n  ignore([\"docs/**\", \"*.md\"])\n\n  group :api do\n    label(\":elixir: API\")\n    scope(:api_code)\n    step(:test, label: \"Test\", command: \"mix test\", timeout_in_minutes: 15)\n    step(:lint, label: \"Lint\", command: \"mix credo\", timeout_in_minutes: 10)\n  end\n\n  group :web do\n    label(\":react: Web\")\n    scope(:web_code)\n    step(:test, label: \"Test\", command: \"pnpm test\", timeout_in_minutes: 15)\n    step(:lint, label: \"Lint\", command: \"pnpm lint\", timeout_in_minutes: 10)\n  end\n\n  group :deploy do\n    label(\":rocket: Deploy\")\n    depends_on([:api, :web])\n    only(\"main\")\n    step(:push, label: \"Push\", command: \"./deploy.sh\")\n  end\nend\n```\n\nCreate a pipeline script at `.buildkite/pipeline.exs`:\n\n```elixir\nMix.install([{:buildkite_pipette, \"~\u003e 0.4\"}])\nPipette.run(MyApp.Pipeline)\n```\n\nWire it into your `.buildkite/pipeline.yml`:\n\n```yaml\nsteps:\n  - label: \":pipeline: Generate\"\n    command: elixir .buildkite/pipeline.exs\n```\n\n## Installation\n\nAdd `pipette` to your `mix.exs` dependencies:\n\n```elixir\ndef deps do\n  [{:buildkite_pipette, \"~\u003e 0.4\"}]\nend\n```\n\nOr use `Mix.install` in standalone pipeline scripts (no project required):\n\n```elixir\nMix.install([{:buildkite_pipette, \"~\u003e 0.4\"}])\n```\n\n## How It Works\n\n```\npipeline.exs\n    |\n    v\nSpark DSL compile\n    |\n    v\n+-------------------+\n| Validate config   |  scope refs, dep refs, cycles, labels (compile-time verifiers)\n+-------------------+\n    |\n    v\n+-------------------+\n| Build context     |  BUILDKITE_BRANCH, BUILDKITE_MESSAGE, etc.\n+-------------------+\n    |\n    v\n+-------------------+\n| Detect changes    |  git diff --name-only \u003cbase\u003e\n+-------------------+\n    |\n    v\n+-------------------+\n| Activation engine |\n|                   |\n| 1. Branch policy  |  main -\u003e all groups, release/* -\u003e specific scopes\n| 2. Targeting      |  [ci:api] in commit message or CI_TARGET env\n| 3. Scope matching |  changed files -\u003e fired scopes -\u003e active groups\n| 4. Force groups   |  FORCE_DEPLOY=true -\u003e [:web, :deploy]\n| 5. Pull deps      |  :deploy depends_on :web -\u003e pull :web in\n| 6. only filter    |  :deploy only: \"main\" -\u003e skip on feature branches\n| 7. Step filter    |  [ci:api/test] -\u003e only run the :test step\n+-------------------+\n    |\n    v\n+-------------------+\n| Serialize YAML    |  groups, steps, triggers -\u003e Buildkite YAML\n+-------------------+\n    |\n    v\nbuildkite-agent pipeline upload\n```\n\nThe activation engine runs through these phases in order. Each phase narrows (or expands) the set of active groups. The final set is serialized to YAML and uploaded to Buildkite.\n\n## Pipeline Definition\n\n### `Pipette.Pipeline`\n\nTop-level configuration struct. Built automatically from `use Pipette.DSL` declarations via `Pipette.Info.to_pipeline/1`.\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `branches` | `[Branch.t()]` | Branch policies controlling activation behavior |\n| `scopes` | `[Scope.t()]` | File-to-scope mappings |\n| `groups` | `[Group.t()]` | Step groups (the units of activation) |\n| `triggers` | `[Trigger.t()]` | Downstream pipeline triggers |\n| `ignore` | `[String.t()]` | Glob patterns for files that should not activate anything |\n| `env` | `map() \\| nil` | Pipeline-level environment variables |\n| `secrets` | `[String.t()] \\| nil` | Secret names to inject |\n| `cache` | `keyword() \\| nil` | Cache configuration |\n| `force_activate` | `%{String.t() =\u003e [atom()] \\| :all}` | Env var -\u003e groups to force-activate |\n\n### `Pipette.Branch`\n\nBranch policy controlling how activation works on matching branches.\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `pattern` | `String.t()` | Branch glob pattern (e.g. `\"main\"`, `\"release/*\"`) |\n| `scopes` | `:all \\| [atom()] \\| nil` | `:all` runs everything; a list restricts to named scopes; `nil` uses file detection |\n| `disable` | `[atom()] \\| nil` | Features to disable (e.g. `[:targeting]`) |\n\n### `Pipette.Scope`\n\nMaps file patterns to a named scope.\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `name` | `atom()` | Unique scope identifier |\n| `files` | `[String.t()]` | Glob patterns that trigger this scope |\n| `exclude` | `[String.t()] \\| nil` | Glob patterns to exclude from matching |\n| `activates` | `:all \\| nil` | When `:all`, any match activates every group |\n\n### `Pipette.Group`\n\nA group of Buildkite steps. Groups are the unit of activation — when a scope fires, its bound group runs.\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `name` | `atom()` | Unique group identifier |\n| `label` | `String.t() \\| nil` | Display label in Buildkite UI |\n| `scope` | `atom() \\| nil` | Scope that activates this group |\n| `depends_on` | `atom() \\| [atom()] \\| nil` | Groups this group depends on |\n| `only` | `String.t() \\| [String.t()] \\| nil` | Branch pattern(s) restricting this group |\n| `steps` | `[Step.t()]` | Command steps in this group |\n\n### `Pipette.Step`\n\nA single Buildkite command step.\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `name` | `atom()` | Unique identifier within the group |\n| `label` | `String.t()` | Display label in Buildkite UI |\n| `command` | `String.t() \\| [String.t()]` | Shell command(s) to run |\n| `timeout_in_minutes` | `pos_integer() \\| nil` | Step timeout |\n| `depends_on` | `atom() \\| {atom(), atom()} \\| list()` | Step-level dependencies |\n| `env` | `map() \\| nil` | Step environment variables |\n| `agents` | `map() \\| nil` | Agent targeting rules |\n| `plugins` | `list() \\| nil` | Buildkite plugins |\n| `retry` | `map() \\| nil` | Retry configuration |\n| `parallelism` | `pos_integer() \\| nil` | Parallel job count |\n| `soft_fail` | `boolean() \\| list() \\| nil` | Soft fail configuration |\n| `artifact_paths` | `String.t() \\| [String.t()] \\| nil` | Artifact upload paths |\n\nSee `Pipette.Step` module docs for the full list of fields.\n\n### `Pipette.Trigger`\n\nFires a downstream Buildkite pipeline.\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `name` | `atom()` | Unique trigger identifier |\n| `label` | `String.t() \\| nil` | Display label |\n| `pipeline` | `String.t()` | Slug of the pipeline to trigger |\n| `depends_on` | `atom() \\| [atom()] \\| nil` | Groups that must complete first |\n| `only` | `String.t() \\| [String.t()] \\| nil` | Branch filter |\n| `build` | `map() \\| nil` | Build parameters to pass |\n| `async` | `boolean() \\| nil` | Don't wait for the triggered build |\n\n## Buildkite Plugin\n\nThis repository doubles as a Buildkite plugin. Instead of adding `pipette` to a Mix project, you can use the plugin directly in your `pipeline.yml`:\n\n```yaml\nsteps:\n  - plugins:\n      - tommeier/pipette#v0.4.7:\n          pipeline: .buildkite/pipeline.exs\n```\n\nThe plugin runs `elixir \u003cpipeline\u003e` — your pipeline script should use `Mix.install` to pull in the `pipette` dependency:\n\n```elixir\n# .buildkite/pipeline.exs\nMix.install([{:buildkite_pipette, \"~\u003e 0.4\"}])\n\ndefmodule MyApp.Pipeline do\n  use Pipette.DSL\n\n  # ... your pipeline definition\nend\n\nPipette.run(MyApp.Pipeline)\n```\n\nRequires Elixir to be installed on the Buildkite agent (or use a Docker-based agent with Elixir available).\n\n## Targeting\n\nTargeting lets developers manually select which groups and steps to run, bypassing file-based scope detection.\n\n### Commit message syntax\n\nPrefix your commit message with `[ci:\u003ctargets\u003e]`:\n\n```\n[ci:api] Fix login bug            # run only the :api group\n[ci:api,web] Update shared types  # run :api and :web groups\n[ci:api/test] Fix flaky test      # run only the :test step in :api\n```\n\n### CI_TARGET environment variable\n\nSet `CI_TARGET` on the build (same syntax without brackets):\n\n```bash\nCI_TARGET=api             # run only :api\nCI_TARGET=api/test        # run only :api :test step\nCI_TARGET=api,web         # run :api and :web\n```\n\nCommit message targets take precedence over `CI_TARGET`.\n\n### Disabling targeting\n\nOn branches where you want to run everything (like `main`), disable targeting in the branch policy:\n\n```elixir\nbranch(\"main\", scopes: :all, disable: [:targeting])\n```\n\nSee the [Targeting guide](guides/targeting.md) for more details.\n\n## Force Activation\n\nForce-activate groups via environment variables, bypassing scope detection and `only` branch filters:\n\n```elixir\nforce_activate(%{\"FORCE_DEPLOY\" =\u003e [:web, :deploy], \"FORCE_ALL\" =\u003e :all})\n```\n\nWhen `FORCE_DEPLOY=true` is set on the build, the `:web` and `:deploy` groups are activated regardless of which files changed or which branch you're on.\n\nDependencies are still pulled in — if `:deploy` depends on `:web`, both will run.\n\n## Dynamic Groups\n\nFor monorepos with dynamic package discovery, use the `extra_groups` option:\n\n```elixir\nPipette.run(MyApp.Pipeline,\n  extra_groups: fn _ctx, _changed_files -\u003e\n    \"packages\"\n    |\u003e File.ls!()\n    |\u003e Enum.filter(\u0026File.dir?(Path.join(\"packages\", \u00261)))\n    |\u003e Enum.map(fn pkg -\u003e\n      %Pipette.Group{\n        name: String.to_atom(pkg),\n        label: \":package: #{pkg}\",\n        key: pkg,\n        steps: [\n          %Pipette.Step{\n            name: :test,\n            label: \"Test\",\n            command: \"cd packages/#{pkg} \u0026\u0026 mix test\",\n            key: \"#{pkg}-test\"\n          }\n        ]\n      }\n    end)\n  end\n)\n```\n\nNote: Extra groups are constructed as plain structs since they're generated at runtime, outside the compile-time DSL.\n\nSee the [Dynamic Groups guide](guides/dynamic-groups.md) for more details.\n\n## Testing Your Pipeline\n\nUse `Pipette.generate/2` in your tests to verify activation logic without uploading to Buildkite:\n\n```elixir\ndefmodule MyApp.PipelineTest do\n  use ExUnit.Case\n\n  test \"API changes activate only the API group\" do\n    {:ok, yaml} = Pipette.generate(MyApp.Pipeline,\n      env: %{\n        \"BUILDKITE_BRANCH\" =\u003e \"feature/login\",\n        \"BUILDKITE_PIPELINE_DEFAULT_BRANCH\" =\u003e \"main\",\n        \"BUILDKITE_COMMIT\" =\u003e \"abc123\",\n        \"BUILDKITE_MESSAGE\" =\u003e \"Add login endpoint\"\n      },\n      changed_files: [\"apps/api/lib/user.ex\"]\n    )\n\n    assert yaml =~ \"api\"\n    refute yaml =~ \"web\"\n  end\n\n  test \"docs-only changes produce no pipeline\" do\n    assert :noop = Pipette.generate(MyApp.Pipeline,\n      env: %{\n        \"BUILDKITE_BRANCH\" =\u003e \"docs/update\",\n        \"BUILDKITE_PIPELINE_DEFAULT_BRANCH\" =\u003e \"main\",\n        \"BUILDKITE_COMMIT\" =\u003e \"abc123\",\n        \"BUILDKITE_MESSAGE\" =\u003e \"Update docs\"\n      },\n      changed_files: [\"docs/guide.md\", \"README.md\"]\n    )\n  end\nend\n```\n\nSee the [Testing guide](guides/testing.md) for more patterns.\n\n## License\n\nMIT - see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftommeier%2Fpipette-buildkite-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftommeier%2Fpipette-buildkite-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftommeier%2Fpipette-buildkite-plugin/lists"}