https://github.com/buildkite/dynamic-pipeline-example
An example pipeline that dynamically generates steps using a shell script.
https://github.com/buildkite/dynamic-pipeline-example
buildkite dynamic-pipelines example pipeline
Last synced: about 1 month ago
JSON representation
An example pipeline that dynamically generates steps using a shell script.
- Host: GitHub
- URL: https://github.com/buildkite/dynamic-pipeline-example
- Owner: buildkite
- License: mit
- Created: 2016-02-17T08:03:43.000Z (about 10 years ago)
- Default Branch: main
- Last Pushed: 2025-07-18T05:47:31.000Z (8 months ago)
- Last Synced: 2025-11-03T07:04:19.265Z (5 months ago)
- Topics: buildkite, dynamic-pipelines, example, pipeline
- Language: Shell
- Homepage: https://buildkite.com/buildkite/dynamic-pipeline-example/builds/latest?branch=main
- Size: 358 KB
- Stars: 18
- Watchers: 24
- Forks: 12
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# Buildkite Dynamic Pipeline Steps Example
[](https://buildkite.com/buildkite/dynamic-pipeline-example/builds/latest?branch=main)
[](https://buildkite.com/new)
This repository is an example [Buildkite](https://buildkite.com/) pipeline that shows how to programmatically generate dynamic steps using a shell script.
👉 **See this example in action:** [buildkite/dynamic-pipeline-example](https://buildkite.com/buildkite/dynamic-pipeline-example/builds/latest?branch=main)
See the full [Getting Started Guide](https://buildkite.com/docs/guides/getting-started) for step-by-step instructions on how to get this running, or try it yourself:
[](https://buildkite.com/new)
## How does it work?
This pipeline starts with a single job that runs:
```bash
.buildkite/pipeline.sh | buildkite-agent pipeline upload
```
The script [.buildkite/pipeline.sh](.buildkite/pipeline.sh) does the following:
* Creates a test step for each subdirectory in [specs](specs/)
* Adds a deploy step if the build is on the `main` branch
For non-main branches build it generates:
```yml
steps:
- command: "specs/controllers/test.sh"
label: "controllers"
- command: "specs/features/test.sh"
label: "features"
- command: "specs/models/test.sh"
label: "models"
```
For a `main` branch build it generates:
```yml
steps:
- command: "specs/controllers/test.sh"
label: "controllers"
- command: "specs/features/test.sh"
label: "features"
- command: "specs/models/test.sh"
label: "models"
- wait
- command: "echo Deploy!"
label: ":rocket:"
```
## More ideas
What else could you do? The possibilities are endless. You can use this technique for custom deploy workflows, QA gates, conditional rollbacks, etc.
Tools like [Jobsworth](https://github.com/saymedia/jobsworth) use dynamic steps to manage complex deployment logic.
## License
See [LICENSE.md](LICENSE.md) (MIT)