{"id":13521459,"url":"https://github.com/avmnu-sng/rspec-tracer","last_synced_at":"2025-04-05T02:10:29.949Z","repository":{"id":42520713,"uuid":"393342542","full_name":"avmnu-sng/rspec-tracer","owner":"avmnu-sng","description":"RSpec Tracer is a specs dependency analyzer, flaky tests detector, tests accelerator, and coverage reporter tool.","archived":false,"fork":false,"pushed_at":"2024-03-12T11:14:36.000Z","size":10112,"stargazers_count":153,"open_issues_count":7,"forks_count":18,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-26T07:01:57.177Z","etag":null,"topics":["coverage","coverage-report","jruby","parallel-tests","rails","rspec","ruby","testing"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/avmnu-sng.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":".github/CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":"support/coverage_setup.rb","governance":null,"roadmap":null,"authors":null,"dei":null},"funding":{"open_collective":"rspec-tracer"}},"created_at":"2021-08-06T10:30:32.000Z","updated_at":"2024-12-10T10:08:55.000Z","dependencies_parsed_at":"2024-03-15T00:53:42.463Z","dependency_job_id":null,"html_url":"https://github.com/avmnu-sng/rspec-tracer","commit_stats":{"total_commits":59,"total_committers":2,"mean_commits":29.5,"dds":0.06779661016949157,"last_synced_commit":"568c5e763385187d50b16ff1a95a859f9f288d20"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/avmnu-sng%2Frspec-tracer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/avmnu-sng%2Frspec-tracer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/avmnu-sng%2Frspec-tracer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/avmnu-sng%2Frspec-tracer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/avmnu-sng","download_url":"https://codeload.github.com/avmnu-sng/rspec-tracer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246122391,"owners_count":20726823,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coverage","coverage-report","jruby","parallel-tests","rails","rspec","ruby","testing"],"created_at":"2024-08-01T06:00:34.630Z","updated_at":"2025-03-29T01:12:36.879Z","avatar_url":"https://github.com/avmnu-sng.png","language":"Ruby","funding_links":["https://opencollective.com/rspec-tracer"],"categories":["Ruby","Testing","Tools"],"sub_categories":[],"readme":"![](./readme_files/rspec_tracer.png)\n\n[![Open Source Helpers](https://www.codetriage.com/avmnu-sng/rspec-tracer/badges/users.svg)](https://www.codetriage.com/avmnu-sng/rspec-tracer)\n[![Discord](https://badgen.net/badge/icon/discord?icon=discord\u0026label)](https://discord.gg/H2G9yWeuRZ)\n[![Maintainability](https://api.codeclimate.com/v1/badges/eabce2757839c08d8f8d/maintainability)](https://codeclimate.com/github/avmnu-sng/rspec-tracer/maintainability)\n[![Test Coverage](https://api.codeclimate.com/v1/badges/eabce2757839c08d8f8d/test_coverage)](https://codeclimate.com/github/avmnu-sng/rspec-tracer/test_coverage)\n[![Gem Version](https://badge.fury.io/rb/rspec-tracer.svg)](https://badge.fury.io/rb/rspec-tracer)\n\n![](./readme_files/next_run.gif)\n\nRSpec Tracer is a **specs dependency analyzer**, **flaky tests detector**, **tests accelerator**,\nand **coverage reporter** tool. It maintains a list of files for each test, enabling\nitself to skip tests in the subsequent runs if none of the dependent files are changed.\n\nIt uses [Ruby's built-in coverage library](https://ruby-doc.org/stdlib/libdoc/coverage/rdoc/Coverage.html)\nto keep track of the coverage for each test. For each test executed, the coverage\ndiff provides the desired file list. RSpec Tracer takes care of reporting the\n**correct code coverage when skipping tests** by using the cached reports. Also,\nnote that it will **never skip**:\n\n- **Flaky examples**\n- **Failed examples**\n- **Pending examples**\n\nKnowing the examples and files dependency gives us a better insight into the codebase,\nand we have **a clear idea of what to test for when making any changes**. With this data,\nwe can also analyze the coupling between different components and much more.\n\nRSpec Tracer requires **Ruby 2.5+** and **rspec-core \u003e= 3.6.0**. To use with **Rails 5+**,\nmake sure to use **rspec-rails \u003e= 4.0.0**. If you are using SimpleCov, it is\nrecommended to use **simplecov \u003e= 0.17.0**. To use RSpec Tracer **cache on CI**, you\nneed to have an **S3 bucket** and **[AWS CLI](https://aws.amazon.com/cli/)**\ninstalled.\n\n\u003e You should take some time and go through the **[document](./RSPEC_TRACER.md)**\ndescribing the **intention** and implementation details of **managing dependency**,\n**managing flaky tests**, **skipping tests**, and **caching on CI**.\n\n## Table of Contents\n\n* [Getting Started](#getting-started)\n  * [Working with JRuby](#working-with-jruby)\n  * [Working with Parallel Tests](#working-with-parallel-tests)\n* [Configuring CI](#configuring-ci)\n* [Advanced Configuration](#advanced-configuration)\n  * [Available Settings](#available-settings)\n* [Filters](#filters)\n* [Duplicate Examples](#duplicate-examples)\n* [Demo](#demo)\n\n## Getting Started\n\n1. Add this line to your `Gemfile` and `bundle install`:\n    ```ruby\n    gem 'rspec-tracer', '~\u003e 1.0', group: :test, require: false\n    ```\n\n    And, add the followings to your `.gitignore`:\n    ```\n    rspec_tracer.lock\n    rspec_tracer_cache/\n    rspec_tracer_coverage/\n    rspec_tracer_report/\n    ```\n2. Load and launch RSpec Tracer at the very top of `spec_helper.rb` (or `rails_helper.rb`,\n`test/test_helper.rb`). Note that `RSpecTracer.start` must be issued **before loading\nany of the application code.**\n\n    ```ruby\n    # Load RSpec Tracer\n    require 'rspec_tracer'\n    RSpecTracer.start\n    ```\n\n    **If you are using SimpleCov**, load RSpec Tracer right after the SimpleCov load\n    and launch:\n\n    ```ruby\n    require 'simplecov'\n    SimpleCov.start\n\n    # Load RSpec Tracer\n    require 'rspec_tracer'\n    RSpecTracer.start\n    ```\n\n    If you use RSpec Tracer with SimpleCov, then **SimpleCov would not report branch\n    coverage results even when enabled**.\n\n3. After running your tests, open `rspec_tracer_report/index.html` in the browser\nof your choice.\n\n### Working with JRuby\n\nIt is recommend to use **JRuby 9.2.10.0+**. Also, configure it with **`JRUBY_OPTS=\"--debug -X+O\"`**\nor have the `.jrubyrc` file:\n\n```ruby\ndebug.fullTrace=true\nobjectspace.enabled=true\n```\n\n### Working with Parallel Tests\n\nThe Rspec tracer, by default, supports working with [parallel_tests](https://github.com/grosser/parallel_tests/)\ngem. It maintains a lock file `rspec_tracer.lock` to identify the last\nrunning process. Usually, you are not required to do anything special unless you\ninterrupt the execution in between and the process did not complete correctly.\nIn such a case, you must delete the lock file before the next run.\n\n```sh\nrm -f rspec_tracer.lock \u0026\u0026 bundle exec parallel_rspec\n```\n\n## Configuring CI\n\nTo enable RSpec Tracer to share cache between different builds on CI, update the\nRakefile in your project to have the following:\n```ruby\nspec = Gem::Specification.find_by_name('rspec-tracer')\n\nload \"#{spec.gem_dir}/lib/rspec_tracer/remote_cache/Rakefile\"\n```\n\nBefore running tests, download the remote cache using the following rake task:\n```sh\nbundle exec rake rspec_tracer:remote_cache:download\n```\n\nAfter running tests, upload the local cache using the following rake task:\n```sh\nbundle exec rake rspec_tracer:remote_cache:upload\n```\n\nYou must set the following environment variables:\n\n- **`GIT_DEFAULT_BRANCH`** is the default branch name for the repo, e.g., `main` or `master`.\n\n- **`GIT_BRANCH`** is the git branch name you are running the CI build on.\n\n- **`TEST_SUITES`** is the total number of different test suites you are running.\n  ```sh\n  export TEST_SUITES=8\n  ```\n\n- **`TEST_SUITE_ID`** is the CI build ID. If you have a large set of tests to run,\nit is recommended to run them in separate groups. This way, RSpec Tracer is not\noverwhelmed with loading massive cached data in the memory. Also, it generates and\nuses cache for specific test suites and not merge them.\n  ```sh\n  TEST_SUITE_ID=1 bundle exec rspec spec/models\n  TEST_SUITE_ID=2 bundle exec rspec spec/helpers\n  ```\n\n## Advanced Configuration\n\nConfiguration settings must be defined in **`.rspec-tracer`** file:\n```ruby\nRSpecTracer.configure do\n  config_option 'foo'\nend\n```\n\nYou can additionally define a global config file `~/.rspec-tracer` to share the\ncommon settings across projects.\n\n### Available Settings\n\n- **`root dir`** to set the project root. The default value is the current working\ndirectory.\n\n- **`project_name name`** to set the project name. The default value is the last\npart of the project root directory.\n\n- **`cache_dir dir`** to set the cache directory. The default value is `./rspec_tracer_cache`.\nYou can also set the **`RSPEC_TRACER_CACHE_DIR`** environment variable.\n\n- **`coverage_dir dir`** to set the coverage reports directory. The default is the\n`./rspec_tracer_coverage`. You can also set the **`RSPEC_TRACER_COVERAGE_DIR`**\nenvironment variable.\n\n- **`report_dir dir`** to set the HTML reports directory. The default is the\n`./rspec_tracer_report`.  You can also set the **`RSPEC_TRACER_REPORT_DIR`**\nenvironment variable.\n\n- **`reports_s3_path uri`** to set the AWS S3 URI for all the reports from the current\nrun. You can also set the **`RSPEC_TRACER_REPORTS_S3_PATH`** environment variable.\n\n- **`use_local_aws bool_flag`** to use the `awslocal` AWS CLI with `LocalStack`. You can\nalso set the **`RSPEC_TRACER_USE_LOCAL_AWS`** environment variable.\n\n- **`upload_non_ci_reports bool_flag`** to upload execution reports in a non-CI\nenvironment. You can also set the **`RSPEC_TRACER_UPLOAD_NON_CI_REPORTS`** environment\nvariable.\n\n- **`run_all_examples bool_flag`** to always run all the examples irrespective of cache.\nYou can also set the **`RSPEC_TRACER_RUN_ALL_EXAMPLES`** environment variable.\n\n- **`fail_on_duplicates bool_flag`** to fail with a non-zero exit code in case of\nduplicate examples. The default value is `true`. You can also set the **`RSPEC_TRACER_FAIL_ON_DUPLICATES`** \nenvironment variable.\n\n- **`lock_file file`** to set the lock file when executing with `parallel_tests`. The default\nvalue is `./rspec_tracer.lock`. You can also set the **`RSPEC_TRACER_LOCK_FILE`** environment\nvariable.\n\n- **`log_level level`** to set the log level. The default value is `info`. The possible\nvalues are `off`, `debug`, `info`, `warn`, and `error`. You can also set the\n**`RSPEC_TRACER_LOG_LEVEL`** environment variable.\n\n- **`add_filter filter`** to apply filters on the source files to exclude them\nfrom the dependent files list.\n\n- **`filters.clear`** to remove the configured dependent files filters so far.\n\n- **`add_coverage_filter filter`** to apply filters on the source files to exclude\nthem from the coverage report.\n\n- **`coverage_filters.clear`** to remove the configured coverage files filters so far.\n\n- **`coverage_track_files glob`** to include files in the given glob pattern in\nthe coverage report if these files are not already present.\n\n```ruby\nRSpecTracer.configure do\n  # Configure project root\n  root '/tmp/my_project'\n\n  # Clear existing filters\n  filters.clear\n  # Add dependent files filter\n  add_filter %r{^/tasks/}\n\n  # Clear existing coverage filters\n  coverage_filters.clear\n  # Add coverage files filter\n  add_coverage_filter %w[/features/ /spec/ /tests/]\n\n  # Define glob to track files in the coverage report\n  coverage_track_files '{app,lib}/**/*.rb'\nend\n```\n\n## Filters\n\nBy default, RSpec Tracer ignores all the files outside of the project root directory -\notherwise you would end up with the source files in the gems you are using in the\nproject. It also applies the following filters:\n```ruby\nRSpecTracer.configure do\n  log_level 'warn'\n\n  add_filter '/vendor/bundle/'\n\n  add_coverage_filter %w[\n    /autotest/\n    /features/\n    /spec/\n    /test/\n    /vendor/bundle/\n  ].freeze\nend\n```\n\n### Defining Custom Filteres\n\nYou can currently define a filter using either a String or Regexp (that will then\nbe Regexp-matched against each source file's name relative to the project root),\na block or by passing in your own Filter class.\n\n- **String Filter**: The string filter matches files that have the given string\nin their name. For example, the following string filter will remove all files that\nhave `\"/helpers/\"` in their name.\n  ```ruby\n  RSpecTracer.configure do\n    add_filter '/helpers/'\n  end\n  ```\n\n- **Regex Filter**: The regex filter removes all files that have a successful name\nmatch against the given regex expression. This simple regex filter will remove\nall files that start with `%r{^/helper/}` in their name:\n  ```ruby\n  RSpecTracer.configure do\n    add_filter %r{^/helpers/}\n  end\n  ```\n\n- **Block Filter**: Block filters receive a `Hash` object and expect your block\nto return either **true** (if the file is to be removed from the result) or **false**\n(if the result should be kept). In the below example, the filter will remove all\nfiles that match `\"/helpers/\"` in their path.\n  ```ruby\n  RSpecTracer.configure do\n    add_filter do |source_file|\n      source_file[:file_path].include?('/helpers/')\n    end\n  end\n  ```\n\n  You can also use `source_file[:name]` to define the return value of the block\n  filter for the given source file.\n\n- **Array Filter**: You can pass in an array containing any of the other filter types:\n  ```ruby\n  RSpecTracer.configure do\n    add_filter ['/helpers/', %r{^/utils/}]\n  end\n  ```\n\n## Duplicate Examples\n\nTo uniquely identify the examples is one of the requirements for the correctness\nof the RSpec Tracer. Sometimes, it would not be possible to do so depending upon\nhow we have written the specs. The following attributes determine the uniqueness:\n\n- The example group\n- The example full description\n- The spec file location, i.e., file name and line number\n- All the shared examples and contexts\n\nConsider the following `Calculator` module:\n```ruby\nmodule Calculator\n  module_function\n\n  def add(a, b) a + b; end\n  def sub(a, b) a - b; end\n  def mul(a, b) a * b; end\nend\n```\n\nAnd the corresponding spec file `spec/calculator_spec.rb`:\n```ruby\nRSpec.describe Calculator do\n  describe '#add' do\n    [\n      [1, 2, 3],\n      [0, 0, 0],\n      [5, 32, 37],\n      [-1, -8, -9],\n      [10, -10, 0]\n    ].each { |a, b, r| it { expect(described_class.add(a, b)).to eq(r) } }\n  end\n\n  describe '#sub' do\n    [\n      [1, 2, -1],\n      [10, 0, 10],\n      [37, 5, 32],\n      [-1, -8, 7],\n      [10, 10, 0]\n    ].each do |a, b, r|\n      it 'performs subtraction' do\n        expect(described_class.sub(a, b)).to eq(r)\n      end\n    end\n  end\n\n  describe '#mul' do\n    [\n      [1, 2, -2],\n      [10, 0, 0],\n      [5, 7, 35],\n      [-1, -8, 8],\n      [10, 10, 100]\n    ].each do |a, b, r|\n      it \"multiplies #{a} and #{b} to #{r}\" do\n        expect(described_class.mul(a, b)).to eq(r)\n      end\n    end\n  end\nend\n```\n\nRunning the spec with `bundle exec rspec spec/calculator_spec.rb` generates the\nfollowing output:\n```\nCalculator\n  #mul\n    multiplies 5 and 7 to 35\n    multiplies 10 and 10 to 100\n    multiplies 10 and 0 to 0\n    multiplies 1 and 2 to -2 (FAILED - 1)\n    multiplies -1 and -8 to 8\n  #add\n    example at ./spec/calculator_spec.rb:13\n    example at ./spec/calculator_spec.rb:13\n    example at ./spec/calculator_spec.rb:13\n    example at ./spec/calculator_spec.rb:13\n    example at ./spec/calculator_spec.rb:13\n  #sub\n    performs subtraction\n    performs subtraction\n    performs subtraction\n    performs subtraction\n    performs subtraction\n```\n\nIn this scenario, RSpec Tracer cannot determine the `Calculator#add` and\n`Calculator#sub` group examples.\n\n```\n================================================================================\n   IMPORTANT NOTICE -- RSPEC TRACER COULD NOT IDENTIFY SOME EXAMPLES UNIQUELY\n================================================================================\nRSpec tracer could not uniquely identify the following 10 examples:\n  - Example ID: eabd51a899db4f64d5839afe96004f03 (5 examples)\n      * Calculator#add (spec/calculator_spec.rb:13)\n      * Calculator#add (spec/calculator_spec.rb:13)\n      * Calculator#add (spec/calculator_spec.rb:13)\n      * Calculator#add (spec/calculator_spec.rb:13)\n      * Calculator#add (spec/calculator_spec.rb:13)\n  - Example ID: 72171b502c5a42b9aa133f165cf09ec2 (5 examples)\n      * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)\n      * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)\n      * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)\n      * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)\n      * Calculator#sub performs subtraction (spec/calculator_spec.rb:24)\n```\n\n## Demo\n\n**First Run**\n![](./readme_files/first_run.gif)\n\n**Next Run**\n![](./readme_files/next_run.gif)\n\nYou get the following three reports:\n\n### All Examples Report\n\nThese reports provide basic test information:\n\n**First Run**\n\n![](./readme_files/examples_report_first_run.png)\n\n**Next Run**\n\n![](./readme_files/examples_report_next_run.png)\n\n### Duplicate Examples Report\n\nThese reports provide duplicate tests information.\n\n![](./readme_files/duplicate_examples_report.png)\n\n### Flaky Examples Report\n\nThese reports provide flaky tests information. Assuming **the following two tests\nfailed in the first run.**\n\n**Next Run**\n\n![](./readme_files/flaky_examples_report_first_run.png)\n\n**Another Run**\n\n![](./readme_files/flaky_examples_report_next_run.png)\n\n### Examples Dependency Report\n\nThese reports show a list of dependent files for each test.\n\n![](./readme_files/examples_dependency_report.png)\n\n### Files Dependency Report\n\nThese reports provide information on the total number of tests that will run after changing this particular file.\n\n![](./readme_files/files_dependency_report.png)\n\n## Contributing\n\nRead the [contribution guide](https://github.com/avmnu-sng/rspec-tracer/blob/main/.github/CONTRIBUTING.md).\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n\n## Code of Conduct\n\nEveryone interacting in the Rspec Tracer project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [Code of Conduct](https://github.com/avmnu-sng/rspec-tracer/blob/main/.github/CODE_OF_CONDUCT.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Favmnu-sng%2Frspec-tracer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Favmnu-sng%2Frspec-tracer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Favmnu-sng%2Frspec-tracer/lists"}