Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/tmm1/test-queue
parallel test runner for CI environments
https://github.com/tmm1/test-queue
Last synced: about 2 months ago
JSON representation
parallel test runner for CI environments
- Host: GitHub
- URL: https://github.com/tmm1/test-queue
- Owner: tmm1
- License: mit
- Created: 2013-04-23T20:30:57.000Z (over 11 years ago)
- Default Branch: master
- Last Pushed: 2023-12-30T15:43:16.000Z (9 months ago)
- Last Synced: 2024-07-18T19:13:32.163Z (2 months ago)
- Language: Ruby
- Size: 311 KB
- Stars: 572
- Watchers: 13
- Forks: 74
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-ruby-toolbox - test-queue - minitest/rspec parallel test runner for CI environments (Testing / Distributed Testing)
- awesome-rspec - test-queue - Parallel test runner for CI environments (Parallel execution)
README
# test-queue
[](https://rubygems.org/gems/test-queue)
[](https://github.com/tmm1/test-queue/actions/workflows/test.yml)
Yet another parallel test runner, built using a centralized queue to ensure
optimal distribution of tests between workers.
Specifically optimized for CI environments: build statistics from each run
are stored locally and used to sort the queue at the beginning of the
next run.
## Usage
test-queue bundles `testunit-queue`, `minitest-queue`, and `rspec-queue` binaries which can be used directly:
```console
$ minitest-queue $(find test/ -name \*_test.rb)
$ rspec-queue --format progress spec
```
But the underlying `TestQueue::Runner::TestUnit`, `TestQueue::Runner::Minitest`, and `TestQueue::Runner::RSpec` are
built to be subclassed by your application. I recommend checking a new
executable into your project using one of these superclasses.
```console
$ vim script/test-queue
$ chmod +x script/test-queue
$ git add script/test-queue
```
Since test-queue uses `fork(2)` to spawn off workers, you must ensure each worker
runs in an isolated environment. Use the `after_fork` hook with a custom
runner to reset any global state.
``` ruby
#!/usr/bin/env ruby
class MyAppTestRunner < TestQueue::Runner::Minitest
def after_fork(num)
# Use separate mysql database (we assume it exists and has the right schema already)
ActiveRecord::Base.configurations.configs_for(env_name: 'test', name: 'primary').database << num.to_s
ActiveRecord::Base.establish_connection(:test)
# Use separate redis database
$redis.client.db = num
$redis.client.reconnect
end
def prepare(concurrency)
# Create mysql databases exists with correct schema
concurrency.times do |i|
# ...
end
# If this is a remote master, tell the central master something about us
@remote_master_message = "Output for remote master 123: http://myhost.com/build/123"
end
def around_filter(suite)
$stats.timing("test.#{suite}.runtime") do
yield
end
end
end
MyAppTestRunner.new.execute
```
## Environment variables
- `TEST_QUEUE_WORKERS`: Number of workers to use per master (default: all available cores)
- `TEST_QUEUE_VERBOSE`: Show results as they are available (default: `0`)
- `TEST_QUEUE_SOCKET`: Unix socket `path` (or TCP `address:port` pair) used for communication (default: `/tmp/test_queue_XXXXX.sock`)
- `TEST_QUEUE_RELAY`: Relay results back to a central master, specified as TCP `address:port`
- `TEST_QUEUE_STATS`: `path` to cache build stats in-build CI runs (default: `.test_queue_stats`)
- `TEST_QUEUE_FORCE`: Comma separated list of suites to run
- `TEST_QUEUE_RELAY_TIMEOUT`: When using distributed builds, the amount of time a remote master will try to reconnect to start work
- `TEST_QUEUE_RELAY_TOKEN`: When using distributed builds, this must be the same on remote masters and the central master for remote masters to be able to connect.
- `TEST_QUEUE_REMOTE_MASTER_MESSAGE`: When using distributed builds, set this on a remote master and it will appear in that master's connection message on the central master.
- `TEST_QUEUE_SPLIT_GROUPS`: Split tests up by example rather than example group. Faster for tests with short setup time such as selenium. RSpec only. Add the `:no_split` tag to `ExampleGroups` you don't want split.
## Design
test-queue uses a simple master + pre-fork worker model. The master
exposes a Unix domain socket server which workers use to grab tests off
the queue.
```console
─┬─ 21232 minitest-queue master
├─── 21571 minitest-queue worker [3] - AuthenticationTest
├─── 21568 minitest-queue worker [2] - ApiTest
├─── 21565 minitest-queue worker [1] - UsersControllerTest
└─── 21562 minitest-queue worker [0] - UserTest
```
test-queue also has a distributed mode, where additional masters can share
the workload and relay results back to a central master.
## Distributed mode
To use distributed mode, the central master must listen on a TCP port. Additional masters can be booted
in relay mode to connect to the central master. Remote masters must provide a `TEST_QUEUE_RELAY_TOKEN`
to match the central master's.
```console
$ TEST_QUEUE_RELAY_TOKEN=123 TEST_QUEUE_SOCKET=0.0.0.0:12345 bundle exec minitest-queue ./test/example_test.rb
$ TEST_QUEUE_RELAY_TOKEN=123 TEST_QUEUE_RELAY=0.0.0.0:12345 bundle exec minitest-queue ./test/example_test.rb
$ TEST_QUEUE_RELAY_TOKEN=123 ./test-multi.sh
```
See the [Parameterized Trigger Plugin](https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Trigger+Plugin)
for a simple way to do this with Jenkins.
## See also
- https://github.com/Shopify/rails_parallel
- https://github.com/grosser/parallel_tests