{"id":14969304,"url":"https://github.com/knapsackpro/knapsack","last_synced_at":"2025-12-17T00:03:43.183Z","repository":{"id":18245097,"uuid":"21396060","full_name":"KnapsackPro/knapsack","owner":"KnapsackPro","description":"Knapsack splits tests evenly across parallel CI nodes to run fast CI build and save you time.","archived":false,"fork":false,"pushed_at":"2024-10-17T10:54:52.000Z","size":1094,"stargazers_count":526,"open_issues_count":0,"forks_count":97,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-05-12T12:04:43.101Z","etag":null,"topics":["auto-balancing","buildkite","ci-server","circle-ci","cucumber","minitest","parallelism","rspec","ruby","semaphore-ci","snap-ci","spinach","testing-tools","travis-ci","turnip"],"latest_commit_sha":null,"homepage":"https://knapsackpro.com","language":"Ruby","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KnapsackPro.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-07-01T17:12:27.000Z","updated_at":"2025-04-17T06:09:34.000Z","dependencies_parsed_at":"2023-01-13T19:44:13.654Z","dependency_job_id":"a7e81a1d-0057-4f59-8457-d6962b5cacbd","html_url":"https://github.com/KnapsackPro/knapsack","commit_stats":{"total_commits":631,"total_committers":41,"mean_commits":"15.390243902439025","dds":"0.18700475435816166","last_synced_commit":"cfa60892a34b1007f961b4c85ea9fd1aa08460b1"},"previous_names":["arturt/knapsack"],"tags_count":59,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KnapsackPro%2Fknapsack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KnapsackPro%2Fknapsack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KnapsackPro%2Fknapsack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KnapsackPro%2Fknapsack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KnapsackPro","download_url":"https://codeload.github.com/KnapsackPro/knapsack/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254120179,"owners_count":22017953,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["auto-balancing","buildkite","ci-server","circle-ci","cucumber","minitest","parallelism","rspec","ruby","semaphore-ci","snap-ci","spinach","testing-tools","travis-ci","turnip"],"created_at":"2024-09-24T13:41:34.031Z","updated_at":"2025-12-17T00:03:43.148Z","avatar_url":"https://github.com/KnapsackPro.png","language":"Ruby","readme":"\u003e [!WARNING]\n\u003e Knapsack is [archived](https://knapsackpro.com/knapsack_gem?utm_source=github\u0026utm_medium=readme\u0026utm_campaign=knapsack_gem_archived\u0026utm_content=warning_knapsack_gem). But [Knapsack Pro](https://knapsackpro.com?utm_source=github\u0026utm_medium=readme\u0026utm_campaign=knapsack_gem_archived\u0026utm_content=warning_knapsack_pro) is available.\n\u003e\n\u003e Knapsack Pro comes with a free plan and discounts on paid plans for people coming from Knapsack (see [how to migrate in 10 minutes](./MIGRATE_TO_KNAPSACK_PRO.md)).\n\u003e\n\u003e This repository remains available to fork and the gem hosted on RubyGems, so your existing setup won't be affected.\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://knapsackpro.com?utm_source=github\u0026utm_medium=readme\u0026utm_campaign=knapsack_gem_archived\u0026utm_content=hero_logo\"\u003e\n    \u003cimg alt=\"Knapsack\" src=\"./.github/assets/knapsack-diamonds.png\" width=\"300\" height=\"300\" style=\"max-width: 100%;\" /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\u003ch3 align=\"center\"\u003eSpeed up your tests\u003c/h3\u003e\n\u003cp align=\"center\"\u003eRun your 1-hour test suite in 2 minutes with optimal parallelisation on your existing CI infrastructure\u003c/p\u003e\n\n---\n\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://rubygems.org/gems/knapsack\"\u003e\n    \u003cimg alt=\"Gem Version\" src=\"https://badge.fury.io/rb/knapsack.svg\" /\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n\u003cbr /\u003e\n\u003cbr /\u003e\n\nKnapsack wraps your current test runner and works with your existing CI infrastructure to split tests optimally.\n\nIt comes in two flavors, `knapsack` and `knapsack_pro`:\n\n|                                 | `knapsack` | `knapsack_pro`                          |\n| ------------------------------- | ---------- | --------------------------------------- |\n| Free                            | ✅         | ✅ [Free plan](https://knapsackpro.com?utm_source=github\u0026utm_medium=readme\u0026utm_campaign=knapsack_gem_archived\u0026utm_content=free_plan) |\n| Static split                    | ✅         | ✅                                      |\n| [Dynamic split](https://docs.knapsackpro.com/overview/#queue-mode-dynamic-split)       | ❌ | ✅ |\n| [Split by test examples](https://docs.knapsackpro.com/ruby/split-by-test-examples/)    | ❌ | ✅ |\n| Graphs, metrics, and stats      | ❌         | ✅                                      |\n| Programming languages           | 🤞 (Ruby)  | ✅ (Ruby, Cypress, Jest, SDK/API)       |\n| CI providers                    | 🤞 Limited | ✅ (All)                                |\n| [Heroku add-on](https://elements.heroku.com/addons/knapsack-pro)                       | ❌ | ✅ |\n| Automated execution time recording                                                     | ❌ | ✅ |\n| Test split based on most recent execution times                                        | ❌ | ✅ |\n| Support for spot/preemptible CI nodes                                                   | ❌ | ✅ |\n| Additional features             | ❌         | 🤘 ([Overview](https://docs.knapsackpro.com/overview/)) |\n|                                 | [Install](#knapsack) | [Install](https://docs.knapsackpro.com) |\n\n## Migrate from `knapsack` to `knapsack_pro`\n\nIf you already use `knapsack` and want to give `knapsack_pro` a try, here's [how to migrate in 10 minutes](./MIGRATE_TO_KNAPSACK_PRO.md).\n\n## `knapsack`\n\nKnapsack generates a test time execution report and uses it for future test runs.\n\nThe `knapsack` gem supports:\n\n* [RSpec](http://rspec.info)\n* [Cucumber](https://cucumber.io)\n* [Minitest](http://docs.seattlerb.org/minitest/)\n* [Spinach](https://github.com/codegram/spinach)\n* [Turnip](https://github.com/jnicklas/turnip)\n\n### Without Knapsack - bad test suite split\n\n![Unbalanced CI nodes without Knapsack gem](./.github/assets/without_knapsack.png)\n\n### With Knapsack - better test suite split\n\n![Balanced CI nodes with Knapsack gem](./.github/assets/with_knapsack.png)\n\n### Requirements\n\n`\u003e= Ruby 2.1.0`\n\n---\n\n\u003c!-- START doctoc generated TOC please keep comment here to allow auto update --\u003e\n\u003c!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --\u003e\n\n- [Update](#update)\n- [Installation](#installation)\n- [Usage](#usage)\n  - [Step for RSpec](#step-for-rspec)\n  - [Step for Cucumber](#step-for-cucumber)\n  - [Step for Minitest](#step-for-minitest)\n  - [Step for Spinach](#step-for-spinach)\n  - [Custom configuration](#custom-configuration)\n  - [Common step](#common-step)\n    - [Adding or removing tests](#adding-or-removing-tests)\n- [Set up your CI server](#set-up-your-ci-server)\n  - [Info about ENV variables](#info-about-env-variables)\n  - [Passing arguments to the Rake task](#passing-arguments-to-the-rake-task)\n    - [Passing arguments to RSpec](#passing-arguments-to-rspec)\n    - [Passing arguments to Cucumber](#passing-arguments-to-cucumber)\n    - [Passing arguments to Minitest](#passing-arguments-to-minitest)\n    - [Passing arguments to Spinach](#passing-arguments-to-spinach)\n  - [Knapsack binary](#knapsack-binary)\n  - [CircleCI](#circleci)\n    - [Step 1](#step-1)\n    - [Step 2](#step-2)\n  - [Travis](#travis)\n    - [Step 1](#step-1-1)\n    - [Step 2](#step-2-1)\n  - [Semaphore](#semaphore)\n    - [Step 1](#step-1-2)\n    - [Step 2](#step-2-2)\n      - [Semaphore 2.0](#semaphore-20)\n  - [Buildkite](#buildkite)\n    - [Step 1](#step-1-3)\n    - [Step 2](#step-2-3)\n  - [GitLab CI](#gitlab-ci)\n    - [Step 1](#step-1-4)\n    - [Step 2](#step-2-4)\n  - [Info for Jenkins](#info-for-jenkins)\n  - [Info for BitBucket Pipelines](#info-for-bitbucket-pipelines)\n    - [Step 1](#step-1-5)\n    - [Step 2](#step-2-5)\n- [FAQ](#faq)\n  - [What does time offset warning mean?](#what-does-time-offset-warning-mean)\n  - [How to generate the Knapsack report?](#how-to-generate-the-knapsack-report)\n  - [What does \"leftover specs\" mean?](#what-does-leftover-specs-mean)\n  - [Why are there \"leftover specs\" after I generate a new report?](#why-are-there-leftover-specs-after-i-generate-a-new-report)\n  - [How can I run tests from multiple directories?](#how-can-i-run-tests-from-multiple-directories)\n  - [How to update the existing Knapsack report for a few test files?](#how-to-update-the-existing-knapsack-report-for-a-few-test-files)\n  - [How to run tests for particular CI node in your development environment](#how-to-run-tests-for-particular-ci-node-in-your-development-environment)\n  - [How can I change the log level?](#how-can-i-change-the-log-level)\n- [Gem tests](#gem-tests)\n  - [Spec](#spec)\n  - [Spec examples](#spec-examples)\n- [Acknowledgements](#acknowledgements)\n- [Mentions](#mentions)\n\n\u003c!-- END doctoc generated TOC please keep comment here to allow auto update --\u003e\n\n\n## Update\n\nPlease check [CHANGELOG.md](./CHANGELOG.md) before updating the gem. Knapsack follows [semantic versioning](http://semver.org).\n\n## Installation\n\nAdd these lines to your application's Gemfile:\n\n```ruby\ngroup :test, :development do\n  gem 'knapsack'\nend\n```\n\nAnd then execute:\n\n```sh\nbundle\n```\n\nAdd this line at the bottom of `Rakefile`:\n\n```ruby\nKnapsack.load_tasks if defined?(Knapsack)\n```\n\n## Usage\n\nHere's an example of a Rails app with Knapsack.\n\n[https://github.com/KnapsackPro/rails-app-with-knapsack](https://github.com/KnapsackPro/rails-app-with-knapsack)\n\n### Step for RSpec\n\nAdd at the beginning of your `spec_helper.rb`:\n\n```ruby\nrequire 'knapsack'\n\n# CUSTOM_CONFIG_GOES_HERE\n\nKnapsack::Adapters::RSpecAdapter.bind\n```\n\n### Step for Cucumber\n\nCreate `features/support/knapsack.rb`:\n\n```ruby\nrequire 'knapsack'\n\n# CUSTOM_CONFIG_GOES_HERE\n\nKnapsack::Adapters::CucumberAdapter.bind\n```\n\n### Step for Minitest\n\nAdd the Knapsack code after you load the app environment in `test/test_helper.rb`:\n\n```ruby\nENV['RAILS_ENV'] ||= 'test'\nrequire File.expand_path('../../config/environment', __FILE__)\nrequire 'rails/test_help'\n\nrequire 'knapsack'\n\n# CUSTOM_CONFIG_GOES_HERE\n\nknapsack_adapter = Knapsack::Adapters::MinitestAdapter.bind\nknapsack_adapter.set_test_helper_path(__FILE__)\n```\n\n### Step for Spinach\n\nCreate `features/support/env.rb`:\n\n```ruby\nrequire 'knapsack'\n\n# CUSTOM_CONFIG_GOES_HERE\n\nKnapsack::Adapters::SpinachAdapter.bind\n```\n\n### Custom configuration\n\nYou can change the default Knapsack configuration for RSpec, Cucumber, Minitest, or Spinach tests.\n\nHere are some examples (that you can insert in `CUSTOM_CONFIG_GOES_HERE`):\n\n```ruby\nKnapsack.tracker.config({\n  enable_time_offset_warning: true,\n  time_offset_in_seconds: 30\n})\n\nKnapsack.report.config({\n  test_file_pattern: 'spec/**{,/*/**}/*_spec.rb', # default value based on adapter\n  report_path: 'knapsack_custom_report.json'\n})\n\n# You can use your logger:\nrequire 'logger'\nKnapsack.logger = Logger.new(STDOUT)\nKnapsack.logger.level = Logger::INFO\n```\n\n### Common step\n\nGenerate the time execution report for your test files. Run the command below on one of your CI nodes:\n\n```sh\n# Step for RSpec:\nKNAPSACK_GENERATE_REPORT=true bundle exec rspec spec\n\n# Step for Cucumber:\nKNAPSACK_GENERATE_REPORT=true bundle exec cucumber features\n\n# Step for Minitest:\nKNAPSACK_GENERATE_REPORT=true bundle exec rake test\n\n# If you use Rails 5.0.x then run this instead:\nKNAPSACK_GENERATE_REPORT=true bundle exec rake test\n\n# If you use Rails \u003e= 5.1's SystemTest, run both unit and system tests:\nKNAPSACK_GENERATE_REPORT=true bundle exec rake test test:system\n\n# Step for Spinach:\nKNAPSACK_GENERATE_REPORT=true bundle exec spinach\n```\n\nCommit the generated report `knapsack_rspec_report.json`, `knapsack_cucumber_report.json`, `knapsack_minitest_report.json` or `knapsack_spinach_report.json` into your repository.\n\nThis report should be updated after you add a lot of new slow tests or you change existing ones, which causes a big time execution difference between CI nodes.\n\nYou will get a time offset warning at the end of the RSpec/Cucumber/Minitest run, which reminds you when it’s a good time to regenerate the Knapsack report.\n\n`KNAPSACK_GENERATE_REPORT` is truthy with `\"true\"` or `0`.  All other values are falsy, though [`\"false\"` and `1` are semantically preferrable](https://en.wikipedia.org/wiki/True_and_false_(commands)).\n\n#### Adding or removing tests\n\nThere is no need to regenerate the report every time you add/remove test files.\n\nIf you remove a test file, Knapsack will ignore its entry in the report. If you add a new test file that is not listed in the report, the test file will be assigned to one of the CI nodes.\n\nYou'll want to regenerate your execution report whenever you remove or add a test file with a long time execution time that would affect one of the CI nodes. Knapsack warns you when it's a good time to regenerate the report.\n\n## Set up your CI server\n\nOn your CI server, run the following command for the first CI node (increase `CI_NODE_INDEX` for the next nodes):\n\n```sh\n# Step for RSpec:\nCI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:rspec\n\n# Step for Cucumber:\nCI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:cucumber\n\n# Step for Minitest:\nCI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:minitest\n\n# Step for Spinach:\nCI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:spinach\n```\n\nYou can add `KNAPSACK_TEST_FILE_PATTERN` if your tests are not in the default directory:\n\n```sh\n# Step for RSpec:\nKNAPSACK_TEST_FILE_PATTERN=\"directory_with_specs/**{,/*/**}/*_spec.rb\" CI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:rspec\n\n# Step for Cucumber:\nKNAPSACK_TEST_FILE_PATTERN=\"directory_with_features/**/*.feature\" CI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:cucumber\n\n# Step for Minitest:\nKNAPSACK_TEST_FILE_PATTERN=\"directory_with_tests/**{,/*/**}/*_spec.rb\" CI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:minitest\n\n# Step for Spinach:\nKNAPSACK_TEST_FILE_PATTERN=\"directory_with_features/**/*.feature\" CI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:spinach\n```\n\nYou can set `KNAPSACK_REPORT_PATH` if your Knapsack report was saved in a non-default location:\n\n```sh\n# Step for RSpec:\nKNAPSACK_REPORT_PATH=\"knapsack_custom_report.json\" CI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:rspec\n\n# Step for Cucumber:\nKNAPSACK_REPORT_PATH=\"knapsack_custom_report.json\" CI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:cucumber\n\n# Step for Minitest:\nKNAPSACK_REPORT_PATH=\"knapsack_custom_report.json\" CI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:minitest\n\n# Step for Spinach:\nKNAPSACK_REPORT_PATH=\"knapsack_custom_report.json\" CI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:spinach\n```\n\n### Info about ENV variables\n\n`CI_NODE_TOTAL` - total number of CI nodes you have.\n\n`CI_NODE_INDEX` - index of the current CI node starting from 0 (ie, the second CI node should have `CI_NODE_INDEX=1`).\n\nSome CI providers like GitLab CI have the same name of environment variable like `CI_NODE_INDEX`, which starts from 1 instead of 0. Knapsack will automatically pick it up and change it from 1 to 0.\n\n### Passing arguments to the Rake task\n\n#### Passing arguments to RSpec\n\nKnapsack allows you to pass arguments through to RSpec. For example, if you want to run only specs that have the tag `focus`. If you do this with RSpec directly, it would look like:\n\n```sh\nbundle exec rake rspec --tag focus\n```\n\nTo do this with Knapsack, you simply add your RSpec arguments as parameters to the Knapsack Rake task:\n\n```sh\nbundle exec rake \"knapsack:rspec[--tag focus]\"\n```\n\nRemember that using tags to limit which specs get run will affect the time each file takes to run. One solution to this is to generate a new `knapsack_rspec_report.json` for the commonly run scenarios.\n\n#### Passing arguments to Cucumber\n\n```sh\nbundle exec rake \"knapsack:cucumber[--name feature]\"\n```\n\n#### Passing arguments to Minitest\n\n```sh\nbundle exec rake \"knapsack:minitest[--arg_name value]\"\n```\n\nFor instance, to run verbose tests:\n\n```sh\nbundle exec rake \"knapsack:minitest[--verbose]\"\n```\n\n#### Passing arguments to Spinach\n\n```sh\nbundle exec rake \"knapsack:spinach[--name feature]\"\n```\n\n### Knapsack binary\n\nYou can install `knapsack` globally and use the binary:\n\n```sh\nknapsack rspec \"--tag custom_tag_name --profile\"\nknapsack cucumber\nknapsack minitest \"--verbose --pride\"\nknapsack spinach \"-f spinach_examples\"\n```\n\nHere's an [example](https://github.com/KnapsackPro/knapsack/pull/21) when it might be useful.\n\n### CircleCI\n\nIf you are using circleci.com, you can omit `CI_NODE_TOTAL` and `CI_NODE_INDEX`. Knapsack will use the `CIRCLE_NODE_TOTAL` and `CIRCLE_NODE_INDEX` provided by CircleCI.\n\nHere is an example for test configuration in your `.circleci/config.yml` file:\n\n#### Step 1\n\nRun all the tests on a single CI node with the enabled report generator:\n\n```yaml\n# CircleCI 2.0\n- run:\n  name: Step for RSpec\n  command: |\n    # export is important here\n    export KNAPSACK_GENERATE_REPORT=true\n    bundle exec rspec spec\n\n- run:\n  name: Step for Cucumber\n  command: |\n    # export is important here\n    export KNAPSACK_GENERATE_REPORT=true\n    bundle exec cucumber features\n\n- run:\n  name: Step for Minitest\n  command: |\n    # export is important here\n    export KNAPSACK_GENERATE_REPORT=true\n    bundle exec rake test\n    # For Rails 5.1 runs unit and system tests\n    bundle exec rake test test:system\n\n- run:\n  name: Step for Spinach\n  command: |\n    # export is important here\n    export KNAPSACK_GENERATE_REPORT=true\n    bundle exec rspec spinach\n```\n\nAfter the tests pass, you should copy the Knapsack JSON report and commit it into your repository as `knapsack_rspec_report.json`, `knapsack_cucumber_report.json`, `knapsack_minitest_report.json` or `knapsack_spinach_report.json`.\n\n#### Step 2\n\nUpdate the test command and enable parallelism (remember to add additional containers for your project in the CircleCI settings):\n\n```yaml\n# CircleCI 2.0\n- run:\n  name: Step for RSpec\n  command: bundle exec rake knapsack:rspec\n\n- run:\n  name: Step for Cucumber\n  command: bundle exec rake knapsack:cucumber\n\n- run:\n  name: Step for Minitest\n  command: bundle exec rake knapsack:minitest\n\n- run:\n  name: Step for Spinach\n  command: bundle exec rake knapsack:spinach\n```\n\n### Travis\n\n#### Step 1\n\nRun all the tests on a single CI node with the enabled report generator. Edit `.travis.yml`:\n\n```yaml\nscript:\n  # Step for RSpec:\n  - \"KNAPSACK_GENERATE_REPORT=true bundle exec rspec spec\"\n\n  # Step for Cucumber:\n  - \"KNAPSACK_GENERATE_REPORT=true bundle exec cucumber features\"\n\n  # Step for Minitest:\n  - \"KNAPSACK_GENERATE_REPORT=true bundle exec rake test\"\n  - \"KNAPSACK_GENERATE_REPORT=true bundle exec rake test test:system\" # For Rails 5.1 runs unit and system tests\n\n  # Step for Spinach:\n  - \"KNAPSACK_GENERATE_REPORT=true bundle exec spinach\"\n```\n\nAfter the tests pass, you should copy the Knapsack JSON report and commit it into your repository as `knapsack_rspec_report.json`, `knapsack_cucumber_report.json`, `knapsack_minitest_report.json` or `knapsack_spinach_report.json`.\n\n#### Step 2\n\nYou can parallelize your builds across virtual machines with the [Travis matrix feature](http://docs.travis-ci.com/user/speeding-up-the-build/#Parallelizing-your-builds-across-virtual-machines). Edit `.travis.yml`:\n\n```yaml\nscript:\n  # Step for RSpec:\n  - \"bundle exec rake knapsack:rspec\"\n\n  # Step for Cucumber:\n  - \"bundle exec rake knapsack:cucumber\"\n\n  # Step for Minitest:\n  - \"bundle exec rake knapsack:minitest\"\n\n  # Step for Spinach:\n  - \"bundle exec rake knapsack:spinach\"\n\nenv:\n  - CI_NODE_TOTAL=2 CI_NODE_INDEX=0\n  - CI_NODE_TOTAL=2 CI_NODE_INDEX=1\n```\n\nIf you want to have both global and matrix ENVs:\n\n```yaml\nscript:\n  # Step for RSpec:\n  - \"bundle exec rake knapsack:rspec\"\n\n  # Step for Cucumber:\n  - \"bundle exec rake knapsack:cucumber\"\n\n  # Step for Minitest:\n  - \"bundle exec rake knapsack:minitest\"\n\n  # Step for Spinach:\n  - \"bundle exec rake knapsack:spinach\"\n\nenv:\n  global:\n    - RAILS_ENV=test\n    - MY_GLOBAL_VAR=123\n    - CI_NODE_TOTAL=2\n  jobs:\n    - CI_NODE_INDEX=0\n    - CI_NODE_INDEX=1\n```\n\nSuch configuration will generate a matrix with the two following rows:\n\n```sh\nCI_NODE_TOTAL=2 CI_NODE_INDEX=0 RAILS_ENV=test MY_GLOBAL_VAR=123\nCI_NODE_TOTAL=2 CI_NODE_INDEX=1 RAILS_ENV=test MY_GLOBAL_VAR=123\n```\n\nMore info about global and matrix ENV configuration in the [Travis docs](http://docs.travis-ci.com/user/build-configuration/#Environment-variables).\n\n### Semaphore\n\n#### Step 1\n\nRun all the tests on a single CI node with the enabled report generator:\n\n```sh\n# Step for RSpec\nKNAPSACK_GENERATE_REPORT=true bundle exec rspec spec\n\n# Step for Cucumber\nKNAPSACK_GENERATE_REPORT=true bundle exec cucumber features\n\n# Step for Minitest\nKNAPSACK_GENERATE_REPORT=true bundle exec rake test\nKNAPSACK_GENERATE_REPORT=true bundle exec rake test test:system # For Rails 5.1 runs unit and system tests\n\n# Step for Spinach\nKNAPSACK_GENERATE_REPORT=true bundle exec spinach\n```\n\nAfter the tests pass, you should copy the Knapsack JSON report and commit it into your repository as `knapsack_rspec_report.json`, `knapsack_cucumber_report.json`, `knapsack_minitest_report.json` or `knapsack_spinach_report.json`.\n\n#### Step 2\n\n##### Semaphore 2.0\n\nKnapsack supports the environment variables provided by Semaphore CI 2.0. Edit `.semaphore/semaphore.yml`:\n\n```yaml\n# .semaphore/semaphore.yml\n\n# Use the latest stable version of Semaphore 2.0 YML syntax:\nversion: v1.0\n\n# Name your pipeline. In case you connect multiple pipelines with promotions,\n# the name will help you differentiate between, for example, a CI build phase\n# and delivery phases.\nname: Demo Rails 5 app\n\n# An agent defines the environment in which your code runs.\n# It is a combination of one of available machine types and operating\n# system images.\n# See https://docs.semaphoreci.com/article/20-machine-types\n# and https://docs.semaphoreci.com/article/32-ubuntu-1804-image\nagent:\n  machine:\n    type: e1-standard-2\n    os_image: ubuntu1804\n\n# Blocks are the heart of a pipeline and are executed sequentially.\n# Each block has a task that defines one or more jobs. Jobs define the\n# commands to execute.\n# See https://docs.semaphoreci.com/article/62-concepts\nblocks:\n  - name: Setup\n    task:\n      env_vars:\n        - name: RAILS_ENV\n          value: test\n      jobs:\n        - name: bundle\n          commands:\n          # Checkout code from Git repository. This step is mandatory if the\n          # job is to work with your code.\n          # Optionally you may use --use-cache flag to avoid roundtrip to\n          # remote repository.\n          # See https://docs.semaphoreci.com/article/54-toolbox-reference#libcheckout\n          - checkout\n          # Restore dependencies from cache.\n          # Read about caching: https://docs.semaphoreci.com/article/54-toolbox-reference#cache\n          - cache restore gems-$SEMAPHORE_GIT_BRANCH-$(checksum Gemfile.lock),gems-$SEMAPHORE_GIT_BRANCH-,gems-master-\n          # Set Ruby version:\n          - sem-version ruby 2.6.1\n          - bundle install --jobs=4 --retry=3 --path vendor/bundle\n          # Store the latest version of dependencies in cache,\n          # to be used in next blocks and future workflows:\n          - cache store gems-$SEMAPHORE_GIT_BRANCH-$(checksum Gemfile.lock) vendor/bundle\n\n  - name: RSpec tests\n    task:\n      env_vars:\n        - name: RAILS_ENV\n          value: test\n        - name: PGHOST\n          value: 127.0.0.1\n        - name: PGUSER\n          value: postgres\n      # This block runs two jobs in parallel and they both share common\n      # setup steps. We can group them in a prologue.\n      # See https://docs.semaphoreci.com/article/50-pipeline-yaml#prologue\n      prologue:\n        commands:\n          - checkout\n          - cache restore gems-$SEMAPHORE_GIT_BRANCH-$(checksum Gemfile.lock),gems-$SEMAPHORE_GIT_BRANCH-,gems-master-\n          # Start Postgres database service.\n          # See https://docs.semaphoreci.com/article/54-toolbox-reference#sem-service\n          - sem-service start postgres\n          - sem-version ruby 2.6.1\n          - bundle install --jobs=4 --retry=3 --path vendor/bundle\n          - bundle exec rake db:setup\n\n      jobs:\n      - name: Run tests with Knapsack\n        parallelism: 2\n        commands:\n          # Step for RSpec:\n          - bundle exec rake knapsack:rspec\n          # Step for Cucumber:\n          - bundle exec rake knapsack:cucumber\n          # Step for Minitest:\n          - bundle exec rake knapsack:minitest\n          # Step for Spinach:\n          - bundle exec rake knapsack:spinach\n```\n\n### Buildkite\n\n#### Step 1\n\nRun all the tests on a single CI node with the enabled report generator. Run the following commands locally:\n\n```sh\n# Step for RSpec:\nKNAPSACK_GENERATE_REPORT=true bundle exec rspec spec\n\n# Step for Cucumber:\nKNAPSACK_GENERATE_REPORT=true bundle exec cucumber features\n\n# Step for Minitest:\nKNAPSACK_GENERATE_REPORT=true bundle exec rake test\nKNAPSACK_GENERATE_REPORT=true bundle exec rake test test:system # For Rails 5.1 runs unit and system tests\n\n# Step for Spinach:\nKNAPSACK_GENERATE_REPORT=true bundle exec spinach\n```\n\nAfter the tests pass, you should copy the Knapsack JSON report and commit it into your repository as `knapsack_rspec_report.json`, `knapsack_cucumber_report.json`, `knapsack_minitest_report.json` or `knapsack_spinach_report.json`.\n\n#### Step 2\n\nKnapsack supports the Buildkite ENVs `BUILDKITE_PARALLEL_JOB_COUNT` and `BUILDKITE_PARALLEL_JOB`. Just configure the parallelism parameter in your build step and run the appropriate command in your build:\n\n```sh\n# Step for RSpec:\nbundle exec rake knapsack:rspec\n\n# Step for Cucumber:\nbundle exec rake knapsack:cucumber\n\n# Step for Minitest:\nbundle exec rake knapsack:minitest\n\n# Step for Spinach:\nbundle exec rake knapsack:spinach\n```\n\nWhen using the `docker-compose` plugin on Buildkite, you have to pass the following environment variables:\n\n```yaml\nsteps:\n  - label: \"Test\"\n    parallelism: 2\n    plugins:\n      - docker-compose#3.0.3:\n        run: app\n        # Use the proper Knapsack command for your test runner:\n        command: bundle exec rake knapsack:rspec\n        config: docker-compose.test.yml\n        env:\n          - BUILDKITE_PARALLEL_JOB_COUNT\n          - BUILDKITE_PARALLEL_JOB\n```\n\n### GitLab CI\n\nIf you are using GitLab \u003e= 11.5, you can omit `CI_NODE_TOTAL` and `CI_NODE_INDEX`. Knapsack will use the `CI_NODE_TOTAL` and `CI_NODE_INDEX` provided by GitLab if you use the [`parallel`](https://docs.gitlab.com/ee/ci/yaml/#parallel) option in GitLab CI.\n\n#### Step 1\n\nRun all the tests on a single CI node with the enabled report generator:\n\n```yaml\ntest:\n  script: KNAPSACK_GENERATE_REPORT=true bundle exec rspec spec\n```\n\nHere are other commands you could use instead of RSpec:\n\n```sh\n# Step for Cucumber\nKNAPSACK_GENERATE_REPORT=true bundle exec cucumber features\n\n# Step for Minitest\nKNAPSACK_GENERATE_REPORT=true bundle exec rake test\nKNAPSACK_GENERATE_REPORT=true bundle exec rake test test:system # For Rails 5.1 runs unit and system tests\n\n# Step for Spinach\nKNAPSACK_GENERATE_REPORT=true bundle exec spinach\n```\n\nAfter the tests pass, you should copy the Knapsack JSON report and commit it into your repository as `knapsack_rspec_report.json`, `knapsack_cucumber_report.json`, `knapsack_minitest_report.json` or `knapsack_spinach_report.json`.\n\n#### Step 2\n\nUpdate test command and [enable parallelism](https://docs.gitlab.com/ee/ci/yaml/#parallel) (remember to set the proper parallel value for your project):\n\n```yaml\ntest:\n  script: bundle exec rake knapsack:rspec\n  parallel: 2\n```\n\nHere are other commands you could use instead of Knapsack for RSpec:\n\n```sh\n# Step for Cucumber\nbundle exec rake knapsack:cucumber\n\n# Step for Minitest\nbundle exec rake knapsack:minitest\n\n# Step for Spinach\nbundle exec rake knapsack:spinach\n```\n\n### Info for Jenkins\n\nTo run parallel jobs with Jenkins you should use Jenkins Pipeline.\n\nYou can learn the basics in [Parallelism and Distributed Builds with Jenkins](https://www.cloudbees.com/blog/parallelism-and-distributed-builds-jenkins).\n\nHere is an example [`Jenkinsfile`](https://github.com/mknapik/jenkins-pipeline-knapsack/blob/master/Jenkinsfile) using Jenkins Pipeline and Knapsack.\n\nMore tips can be found in this [issue](https://github.com/KnapsackPro/knapsack/issues/42).\n\n### Info for BitBucket Pipelines\n\n#### Step 1\n\nRun all the tests on a single CI node with the enabled report generator. Run the following commands locally:\n\n```sh\n# Step for RSpec\nKNAPSACK_GENERATE_REPORT=true bundle exec rspec spec\n\n# Step for Cucumber\nKNAPSACK_GENERATE_REPORT=true bundle exec cucumber features\n\n# Step for Minitest\nKNAPSACK_GENERATE_REPORT=true bundle exec rake test\nKNAPSACK_GENERATE_REPORT=true bundle exec rake test test:system # For Rails 5.1 runs unit and system tests\n\n# Step for Spinach\nKNAPSACK_GENERATE_REPORT=true bundle exec spinach\n```\n\nAfter the tests pass, you should copy the Knapsack JSON report and commit it into your repository as `knapsack_rspec_report.json`, `knapsack_cucumber_report.json`, `knapsack_minitest_report.json` or `knapsack_spinach_report.json`.\n\n#### Step 2\n\nKnapsack supports BitBucket Pipelines ENVs `BITBUCKET_PARALLEL_STEP_COUNT` and `BITBUCKET_PARALLEL_STEP`. Just configure the parallelism parameter in your build step and run the appropriate command in your build:\n\n```sh\n# Step for RSpec:\nbundle exec rake knapsack:rspec\n\n# Step for Cucumber:\nbundle exec rake knapsack:cucumber\n\n# Step for Minitest:\nbundle exec rake knapsack:minitest\n\n# Step for Spinach:\nbundle exec rake knapsack:spinach\n```\n\n## FAQ\n\n### What does time offset warning mean?\n\nAt the end of a test run, you may see the following warning:\n\n```\n========= Knapsack Time Offset Warning ==========\nTime offset: 30s\nMax allowed node time execution: 02m 30s\nExceeded time: 37s\n```\n\n`Time offset: 30s` is the current time offset value (by default it's 30s).\n\nLet’s assume the whole test suite takes 4 minutes, and you split across 2 CI nodes. The optimal split would be 2 minutes per node.\n\nWith `Time offset: 30s`, you'll see a warning to regenerate the Knapsack report when tests on single CI node take longer than 2 minutes and 30s.\n\n`Max allowed node time execution: 02m 30s` is the average time execution of tests per CI node + time offset. In this case, the average tests time execution per CI node is 2 minutes.\n\n`Exceeded time: 37s` means that tests on this particular CI node took 37s longer than `max allowed node time execution`. Sometimes this value is negative when tests are executed faster than `max allowed node time execution`.\n\n### How to generate the Knapsack report?\n\nIf you want to regenerate the report, take a look at [Common step](#common-step).\n\n```sh\nKNAPSACK_GENERATE_REPORT=true bundle exec rspec spec\n```\n\nOn your development machine, the time execution might be different than CI. For this reason, you should generate the report on a single CI node.\n\n### What does \"leftover specs\" mean?\n\nWhen you run your specs with Knapsack, you'll see in the output something like:\n\n```\nReport specs:\nspec/models/user_spec.rb\nspec/controllers/users_controller_spec.rb\n\nLeftover specs:\nspec/models/book_spec.rb\nspec/models/author_spec.rb\n```\n\nThe leftover specs are the ones that don't have recorded time execution.\n\nThe reason might be:\n\n* The test file was added after Knapsack generated the report\n* Empty spec file with no test cases\n\nLeftover specs are distributed across CI nodes based on file name instead of execution time (which is missing).\n\nIf you have many leftover specs, you can [generate the Knapsack report again](#how-to-generate-the-knapsack-report) to improve the test distribution across CI nodes.\n\n### Why are there \"leftover specs\" after I generate a new report?\n\nIf the test file is empty or only contains pending tests, it cannot be recorded and will end up in leftover specs.\n\n### How can I run tests from multiple directories?\n\nThe test file pattern config option supports any glob pattern handled by [`Dir.glob`](http://ruby-doc.org/core-2.2.0/Dir.html#method-c-glob) and can be configured to pull test files from multiple directories.\n\nFor example, you may want to use `\"{spec,engines/**/spec}/**{,/*/**}/*_spec.rb\"`. In this case, the test directory must also be specified manually using the `KNAPSACK_TEST_DIR` environment variable:\n\n```sh\nKNAPSACK_TEST_DIR=spec KNAPSACK_TEST_FILE_PATTERN=\"{spec,engines/**/spec}/**{,/*/**}/*_spec.rb\" bundle exec rake knapsack:rspec\n```\n\n`KNAPSACK_TEST_DIR` will be the default path for RSpec, where `spec_helper.rb` is expected to be found. Ensure you require it in your test files this way:\n\n```ruby\n# Good:\nrequire_relative 'spec_helper'\n\n# Bad - won't work:\nrequire 'spec_helper'\n```\n\n### How to update the existing Knapsack report for a few test files?\n\nYou may want to look at this [monkey patch](https://github.com/KnapsackPro/knapsack/issues/34).\n\n### How to run tests for particular CI node in your development environment\n\nIn your development environment, you can debug tests that were run on a particular CI node:\n\n```sh\nCI_NODE_TOTAL=2 \\\nCI_NODE_INDEX=0 \\\nbundle exec rake \"knapsack:rspec[--seed 123]\"\n```\n\n### How can I change the log level?\n\nYou can change the log level by specifying the `KNAPSACK_LOG_LEVEL` environment variable:\n\n```sh\nKNAPSACK_LOG_LEVEL=warn bundle exec rake knapsack:rspec\n```\n\nAvailable values are `debug`, `info`, and `warn`. The default log level is `info`.\n\n## Gem tests\n\n### Spec\n\nTo run the specs for Knapsack:\n\n```sh\nbundle exec rspec spec\n```\n\n### Spec examples\n\nThe directory `spec_examples` contains examples of fast and slow specs.\n\nTo generate a new Knapsack report for specs with `focus` tag (only the specs in `spec_examples/leftover` have no `focus` tag):\n\n```sh\nKNAPSACK_GENERATE_REPORT=true bundle exec rspec --default-path spec_examples --tag focus\n```\n\n**Warning:** The current `knapsack_rspec_report.json` file was generated for `spec_examples` excluding `spec_examples/leftover/` to see how leftover specs are badly distributed across CI nodes.\n\nTo see specs distributed for the first CI node:\n\n```sh\nCI_NODE_TOTAL=2 CI_NODE_INDEX=0 KNAPSACK_SPEC_PATTERN=\"spec_examples/**{,/*/**}/*_spec.rb\" bundle exec rake knapsack:rspec\n```\n\nSpecs in `spec_examples/leftover` take more than 3 seconds. This should cause a Knapsack time offset warning because we set `time_offset_in_seconds` to 3 in `spec_examples/spec_helper.rb`:\n\n```sh\nbundle exec rspec --default-path spec_examples\n```\n\n## Acknowledgements\n\n[Małgorzata Nowak](https://github.com/informatykgosia) for the beautiful logo.\n\n## Mentions\n\n* Lunar Logic Blog | [Parallel your specs and don’t waste time](http://blog.lunarlogic.io/2014/parallel-your-specs-and-dont-waste-time/)\n* Travis CI | [Parallelizing RSpec and Cucumber on multiple VMs](http://docs.travis-ci.com/user/speeding-up-the-build/#Parallelizing-RSpec-and-Cucumber-on-multiple-VMs)\n* Buildkite | [Libraries](https://buildkite.com/docs/guides/parallelizing-builds#libraries)\n* CircleCI | [Test splitting documentation](https://circleci.com/docs/2.0/parallelism-faster-jobs/#other-ways-to-split-tests)\n* GitLab | [How we used parallel CI/CD jobs to increase our productivity](https://about.gitlab.com/blog/2021/01/20/using-run-parallel-jobs/)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fknapsackpro%2Fknapsack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fknapsackpro%2Fknapsack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fknapsackpro%2Fknapsack/lists"}