{"id":13532894,"url":"https://github.com/jwplayer/buildpipe-buildkite-plugin","last_synced_at":"2025-12-15T04:19:40.344Z","repository":{"id":42526088,"uuid":"120096012","full_name":"jwplayer/buildpipe-buildkite-plugin","owner":"jwplayer","description":":shipit: Dynamically generate Buildkite pipelines based on project changes","archived":false,"fork":false,"pushed_at":"2023-07-17T15:52:08.000Z","size":250,"stargazers_count":91,"open_issues_count":7,"forks_count":24,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-05-21T13:58:05.435Z","etag":null,"topics":["buildkite","buildkite-plugin","cicd","monorepo"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jwplayer.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-02-03T13:59:36.000Z","updated_at":"2024-05-04T18:14:10.000Z","dependencies_parsed_at":"2024-01-14T02:50:42.414Z","dependency_job_id":null,"html_url":"https://github.com/jwplayer/buildpipe-buildkite-plugin","commit_stats":{"total_commits":91,"total_committers":10,"mean_commits":9.1,"dds":0.2857142857142857,"last_synced_commit":"3c081f611f0a98a6ade20ab75f88ee193e8164de"},"previous_names":[],"tags_count":36,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwplayer%2Fbuildpipe-buildkite-plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwplayer%2Fbuildpipe-buildkite-plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwplayer%2Fbuildpipe-buildkite-plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwplayer%2Fbuildpipe-buildkite-plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jwplayer","download_url":"https://codeload.github.com/jwplayer/buildpipe-buildkite-plugin/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247773726,"owners_count":20993639,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["buildkite","buildkite-plugin","cicd","monorepo"],"created_at":"2024-08-01T07:01:14.670Z","updated_at":"2025-12-15T04:19:40.252Z","avatar_url":"https://github.com/jwplayer.png","language":"Go","funding_links":[],"categories":["开发流程工具"],"sub_categories":["CI 工具"],"readme":"Buildpipe\n=========\n\nA Buildkite plugin to dynamically generate pipelines. Especially useful\nfor monorepos where you want to create dependencies between projects.\n\nExample\n-------\n\n![Buildpipe example](images/example.png)\n\n### initial\\_pipeline.yml\n\n```yaml\nsteps:\n  - label: \":buildkite:\"\n    plugins:\n      - jwplayer/buildpipe#v0.10.3:\n          dynamic_pipeline: dynamic_pipeline.yml\n```\n\n### dynamic\\_pipeline.yml\n\n```yaml\nenv:\n  GLOBAL_ENV_VAR: test\nprojects:\n - label: project1\n   path: project1/  # changes in this directory will trigger steps for project1\n   skip:\n     - deploy*  # skip steps with label matching deploy* (e.g. deploy-prd)\n     - test\n   env:\n     PROJECT_ENV_VAR: project1variable\n - label: project2\n   skip: test\n   path:\n      - project2/\n      - project1  # you can trigger a project using multiple paths\n - label: project3\n   skip:\n     - deploy-stg\n   path: project3/somedir/  # subpaths can also be triggered\nsteps:  # the same schema as regular buildkite pipeline steps\n  - label: test\n    env:\n      BUILDPIPE_SCOPE: project  # this variable ensures a test step is generated for each project\n    command:\n      - cd $$BUILDPIPE_PROJECT_PATH  # BUILDPIPE_PROJECT_PATH is set by buildpipe\n      - make test\n  - wait\n  - label: build\n    branches: \"master\"\n    env:\n      BUILDPIPE_SCOPE: project\n    command:\n      - cd $$BUILDPIPE_PROJECT_PATH\n      - make build\n      - make publish-image\n    agents:\n      - queue=build\n  - wait\n  - label: tag\n    branches: \"master\"\n    command:\n      - make tag-release\n  - wait\n  - label: deploy-stg\n    branches: \"master\"\n    concurrency: 1\n    concurrency_group: deploy-stg\n    env:\n      BUILDPIPE_SCOPE: project\n    command:\n      - cd $$BUILDPIPE_PROJECT_PATH\n      - make deploy-staging\n  - wait\n  - block: \":rocket: Release!\"\n    branches: \"master\"\n  - wait\n  - label: deploy-prd\n    branches: \"master\"\n    concurrency: 1\n    concurrency_group: deploy-prd\n    env:\n      BUILDPIPE_SCOPE: project\n    command:\n      - cd $$BUILDPIPE_PROJECT_PATH\n      - make deploy-prod\n```\n\nThe above pipelines specify the following:\n\n-   There are three projects to track in the repository.\n-   The env variable `BUILDPIPE_SCOPE: project` tells buildpipe to\n    generate a step for each project if that project changed.\n-   The `skip` option will skip any step label matching `deploy*`.\n-   The env variable `BUILDPIPE_PROJECT_PATH` is created by buildpipe as\n    the project\\'s path. If multiple paths are specified for a project,\n    it\\'s the first path.\n-   There is a global environment variable defined (`GLOBAL_ENV_VAR`).\n    This variable will be added to every step in the pipeline.\n-   There is also a project scoped environment variable defined (`PROJECT_ENV_VAR`).\n    That variable will be added to all steps of the project where it was defined (`project1`).\n\n### Full working example\n\nFor a full working example, check out [Buildkite Monorepo Example](https://github.com/ksindi/buildkite-monorepo-example).\n\nConfiguration\n-------------\n\n### Plugin\n\n| Option           | Required | Type   | Default | Description\n| ---------------- | -------- | ------ | ------- | -------------------------------------------------- |\n| default_branch   | No       | string | master  | Default branch of repository |\n| diff_pr          | No       | string |         | Override command for non-default branch (see below for a better explanation of the defaults) |\n| diff_default     | No       | string |         | Override command for default branch (see below for a better explanation of the defaults) |\n| dynamic_pipeline | Yes      | string |         | The name including the path to the pipeline that contains all the actual steps |\n| log_level        | No       | string | INFO    | The Level of logging to be used by the python script underneath; pass DEBUG for verbose logging if errors occur |\n\n### Project schema\n\n| Option | Required | Type        | Default | Description                           | Environment variable |\n| ------ | -------- | ----------- | ------- | ------------------------------------- | -------------------- |\n| label  | Yes      | string      |         | Project label                         | `BUILDPIPE_PROJECT_LABEL` |\n| path   | Yes      | array       |         | The path(s) that specify changes to a project | `BUILDPIPE_PROJECT_PATH` |\n| skip   | No       | array       |         | Exclude steps that have labels that match the rule |         |\n| env    | No       | dictionary  |         | Define environment variable on a project scope |         |\n\nOther useful things to note:\n\n-   Option `skip` makes use of Unix shell-style wildcards (Look at\n    .gitignore files for inspiration)\n-   If multiple paths are specified, the environment variable\n    `BUILDPIPE_PROJECT_PATH` will be the first path.\n-   Environment variables are available in the pipeline step.\n\n### Environment Variables\n\nSince version 0.9.2, there is the option to define environment variables on different scope levels in the pipeline.\nBuildkite already supports environment variables defined on a step scope, but buildpipe adds the ability to define global\nand also project specific environment variables.\n\n#### Project env vars\n```yaml\nenv:\n  THIS_IS_A_GLOBAL_ENV_VAR: global\n  GLOBAL_ENV_VAR_2: another_global_value\nprojects:\n  - label: project1\n    path: project1/\n    env:\n      THIS_IS_A_PROJECT_ENV_VAR: project_scoped\n      PROJECT_ENV_VAR_2: second_env_var_value\n  - label: project2\n    ...\nsteps:\n  - label: step1\n    ...\n```\n\n### Specifying build projects\n\nSince version 0.10.0, you can manually specify which projects you wish to build using the `BUILDKITE_PLUGIN_BUILDPIPE_BUILD_PROJECTS` environment variable.\nIf this environemnt variable is provided, it will take precedence over any existing `git` diff. You can specify the build projects in one of two ways:\n\n1. Provide a comma-separated list of build projects (identified by the project label attribute). For example:\n\n```bash\nBUILDKITE_PLUGIN_BUILDPIPE_BUILD_PROJECTS=\"project1,project2\"\n```\n\n2. Build all projects in a Buildpipe configuration:\n\n```bash\nBUILDKITE_PLUGIN_BUILDPIPE_BUILD_PROJECTS=\"*\"\n```\n\n`diff_` commands\n----------------\n\nDepending on your [merge\nstrategy](https://help.github.com/en/github/administering-a-repository/about-merge-methods-on-github),\nyou might need to use different diff command.\n\nBuildpipe assumes you are using a merge strategy on the default branch, which is assumed to be `master`.\n\nThe command for the non-default branch (e.g. when you have a PR up) is:\n```bash\ngit log --name-only --no-merges --pretty=format: origin..HEAD\n```\n\nThe command for the default branch you merge to is currently:\n```bash\ngit log -m -1 --name-only --pretty=format: $BUILDKITE_COMMIT\n```\n\nRequirements\n------------\n\nOnly `curl` is required to download the binary.\n\nInstalling buildpipe\n--------------------\n\nYou can also install buildpipe in the agent bootstrap script:\n\n```bash\ncurl -Lf -o /usr/local/bin/buildpipe https://github.com/jwplayer/buildpipe-buildkite-plugin/releases/download/v${BUILDPIPE_VERSION}/buildpipe-linux \\\n  \u0026\u0026 chmod +x /usr/local/bin/buildpipe\n```\n\nYour initial pipeline would need to pass the options as environment variables with prefix `BUILDKITE_PLUGIN_BUILDPIPE_`. For example:\n\n```yml\nsteps:\n  - label: \"buildpipe\"\n    command: buildpipe\n    env:\n      BUILDKITE_PLUGIN_BUILDPIPE_DYNAMIC_PIPELINE: path/to/dynamic_pipeline.yml\n      BUILDKITE_PLUGIN_BUILDPIPE_LOG_LEVEL: debug\n```\n\nTroubleshooting\n---------------\n\n### Buildpipe is incorrectly showing project as changed\n\nBuildkite doesn\\'t by default do clean checkouts. To enable clean\ncheckouts set the `BUILDKITE_CLEAN_CHECKOUT` [environment variable](https://buildkite.com/docs/pipelines/environment-variables). An\nexample is to modify the pre-checkout hook,\n`.buildkite/hooks/pre-checkout`:\n\n```bash\n#!/bin/bash\nset -euo pipefail\n\necho '--- :house_with_garden: Setting up pre-checkout'\n\nexport BUILDKITE_CLEAN_CHECKOUT=\"true\"\n```\n\nTesting\n-------\n\n```bash\nmake test\n```\n\nLicense\n-------\n\nApache v2\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjwplayer%2Fbuildpipe-buildkite-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjwplayer%2Fbuildpipe-buildkite-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjwplayer%2Fbuildpipe-buildkite-plugin/lists"}