https://github.com/buildkite/rails-parallel-example
An example pipeline that runs Rails tests in parallel across multiple agents using Knapsack.
https://github.com/buildkite/rails-parallel-example
buildkite example knapsack parallel pipeline rails rbenv
Last synced: 4 months ago
JSON representation
An example pipeline that runs Rails tests in parallel across multiple agents using Knapsack.
- Host: GitHub
- URL: https://github.com/buildkite/rails-parallel-example
- Owner: buildkite
- License: mit
- Created: 2016-02-17T03:58:28.000Z (almost 10 years ago)
- Default Branch: main
- Last Pushed: 2025-09-26T04:05:09.000Z (4 months ago)
- Last Synced: 2025-09-29T22:49:59.877Z (4 months ago)
- Topics: buildkite, example, knapsack, parallel, pipeline, rails, rbenv
- Language: Ruby
- Homepage: https://buildkite.com/buildkite/rails-parallel-example/builds/latest?branch=main
- Size: 264 KB
- Stars: 7
- Watchers: 21
- Forks: 13
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# Buildkite Rails Parallel Pipeline Example
[](https://buildkite.com/buildkite/rails-parallel-example/builds/latest?branch=main)
[](https://buildkite.com/new)
This repository is an example [Buildkite](https://buildkite.com/) pipeline that demonstrates how to run 20 parallel test jobs for a [Rails](https://rubyonrails.org/) application using [rbenv](https://github.com/rbenv/rbenv) and [Knapsack](https://github.com/ArturT/knapsack).
👉 **See this example in action:** [buildkite/rails-parallel-example](https://buildkite.com/buildkite/rails-parallel-example/builds/latest?branch=main)
See the full [Getting Started Guide](https://buildkite.com/docs/guides/getting-started) for step-by-step instructions on how to get this running, or try it yourself:
[](https://buildkite.com/new)
## How it works
This repository doesn't include a Rails application, but it does provide the CI setup you'd typically use in a real project.
In the [pipeline configuration file](.buildkite/pipeline.yml) the `parallelism` property for the test step is set to 20. When a build, the step will appear 20 times in the pipeline, each with different environment variables exposed so you can divvy up your test suite accordingly. You can then run 20 agents (on the same machine, or distributed) to work on the 20 jobs in parallel.
This example:
- Uses rbenv to manage Ruby versions
- Runs tests in parallel using Buildkite’s `parallelism` feature
- Leverages [Knapsack](https://github.com/ArturT/knapsack) for test suite splitting
- Can be used on a single machine or distributed across multiple agents
Key files:
- [.buildkite/hooks/environment](.buildkite/hooks/environment): loads rbenv before steps run
- [.buildkite/pipeline.yml](.buildkite/pipeline.yml): sets up parallel steps
- [scripts/ci/setup.sh](scripts/ci/setup.sh): prepares the CI environment
- [scripts/ci/parallel_specs.sh](scripts/ci/parallel_specs.sh): runs specs via Knapsack
### Parallel test execution
The test step in [`pipeline.yml`](.buildkite/pipeline.yml) is configured like this:
```yml
steps:
- name: ":rspec:"
command: "scripts/ci/setup.sh && scripts/ci/parallel_specs.sh"
parallelism: 20
```
Each parallel job receives unique environment variables (`BUILDKITE_PARALLEL_JOB`, `BUILDKITE_PARALLEL_JOB_COUNT`) to help split the workload.
See the [parallelizing builds guide](https://buildkite.com/docs/guides/parallelizing-builds) for more information to create parallelized and distributed builds with Buildkite.
> 💡 Want to use Docker? Try the [Rails Docker Parallel Example](https://github.com/buildkite/rails-docker-parallel-example)
## Knapsack Pro
Consult the [Knapsack documentation](https://github.com/ArturT/knapsack) for configuring your database and dependent services to support running parallel steps on the one machine.
See [Knapsack Pro version of this example pipeline](https://github.com/KnapsackPro/buildkite-rails-parallel-example-with-knapsack_pro) for how to perform dynamic splits/allocation of test nodes.
## License
See [LICENSE.md](LICENSE.md) (MIT)