{"id":20372286,"url":"https://github.com/test-prof/test-prof-autopilot","last_synced_at":"2025-04-12T06:42:44.975Z","repository":{"id":208504900,"uuid":"502209251","full_name":"test-prof/test-prof-autopilot","owner":"test-prof","description":"Automate tests profiling via scenarios","archived":false,"fork":false,"pushed_at":"2023-11-21T20:48:53.000Z","size":109,"stargazers_count":12,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-05-10T22:01:38.954Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/test-prof.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"test-prof"}},"created_at":"2022-06-10T23:55:56.000Z","updated_at":"2024-08-02T13:06:01.281Z","dependencies_parsed_at":null,"dependency_job_id":"d290e59b-8c8b-4666-8b52-990b4ca99311","html_url":"https://github.com/test-prof/test-prof-autopilot","commit_stats":null,"previous_names":["test-prof/test-prof-autopilot"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/test-prof%2Ftest-prof-autopilot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/test-prof%2Ftest-prof-autopilot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/test-prof%2Ftest-prof-autopilot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/test-prof%2Ftest-prof-autopilot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/test-prof","download_url":"https://codeload.github.com/test-prof/test-prof-autopilot/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248530611,"owners_count":21119592,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T01:12:36.936Z","updated_at":"2025-04-12T06:42:44.964Z","avatar_url":"https://github.com/test-prof.png","language":"Ruby","readme":"[![Gem Version](https://badge.fury.io/rb/test-prof-autopilot.svg)](https://rubygems.org/gems/test-prof-autopilot)\n[![Build](https://github.com/test-prof/test-prof-autopilot/workflows/Build/badge.svg)](https://github.com/test-prof/test-prof-autopilot/actions)\n\n# TestProf Autopilot\n\n\u003e 📖 Read also the [\"TestProf III: guided and automated Ruby test profiling\" blog post](https://evilmartians.com/chronicles/test-prof-3-guided-and-automated-ruby-test-profiling).\n\n[TestProf][] has been used by many Ruby/Rails teams to optimize their test suites performance for a while.\n\nUsually, it takes a decent amount of time to profile the test suite initially: we need run many profilers multiple times, tune configuration and sampling parameters. And we repeat this over and over again.\n\nThere are some common patterns in the way we use TestProf, for example: we run StackProf/RubyProf multiple times for different test samples, or we run EventProf for `factory.create` and then use FactoryProf for the slowest tests.\n\nIt seems that there is a room for optimization here: we can automate this common tasks, make robots do all the repetition. And here comes **TestProf Autopilot**!\n\n## Usage\n\nFirst, write a test profiling plan in a Ruby file. For example, here is how you can perform StackProf profiling multiple times against different random subsets and aggregate the results:\n\n```ruby\n# This plan runs multiple test samples and collects StackProf data.\n# The data is aggregated and the top-5 popular methods are displayed in the end.\n#\n# With the help of this plan, you can detect such problems as unnecessary logging/instrumentation in tests,\n# inproper encryption settings, etc.\n#\n# NOTE: `aggregate` takes a block, runs it the specified number of times and merge the reports (i.e., agg_result = prev_result.merge(curr_result))\naggregate(3) { run :stackprof, sample: 100 }\n\n# `report` returns the latest generated report (both `run` and `aggregate` set this value automatically)\n# `#methods` returns the list of collected reports sorted by their popularity\nputs report.methods.take(5)\n```\n\nNow, you can use the `auto-test-prof` command to execute the plan:\n\n```sh\nauto-test-prof -i plan.rb -с \"bundle exec rspec\"\n```\n\nWe specify the base command to run tests via the `-c` option. If you omit the command option, Autopilot would fall back to either `bundle exec rspec` or `bundle exec rake test` depending on the presense of the `spec/` and `test/` directories, respectively.\n\n### Merging results\n\nAutopilot also allows you to merge reports created with it (using the `#save` method). That's useful when you profile tests on CI and want to see the aggregated results. For example, when using TagProf:\n\n```ruby\nrun :tag_prof, events: [\"factory.create\"]\n\nsave report, file_name: \"tag_prof_#{ENV[\"CI_NODE_INDEX\"]}\"\n```\n\nThen, assuming all reports were downloaded:\n\n```sh\n$ auto-test-prof --merge tag_prof --reports tag_prof_*.json\n\nMerging tag_prof reports at tag_prof_1.json, tag_prof_2.json, tag_prof_3.json\n\n[TEST PROF] TagProf report for type\n\n       type          time   factory.create    total  %total   %time           avg\n\n      model     28:08.654        19:58.371     1730   56.44   46.23     00:00.976\n    service     20:56.071        16:14.435      808   29.18   28.35     00:01.554\n        api     04:48.179        03:54.178      214    7.32    4.78     00:01.346\n        ...\n```\n\n### API\n\n- `run(profiler_name, **options)`: launch the test command with the specified profiler activated; options depend on the profiler, but there are some commont: `sample: \u003cnumber\u003e` — enables sampling, `paths: \u003carray of paths\u003e` — adds the list of paths to the command.\n\n- `info(report)`: shows the report in the console\n\n- `save(report, path)`: store\n\n- `aggregate(num_calls) { ... }`: aggregates reports obtained by calling the block `num_calls` times.\n\nYou can find more examples in the [examples/](examples/) folder.\n\n### Supported profilers\n\nCurrently, Autopilot supports the following Test Prof profilers:\n\n- [EventProf](https://test-prof.evilmartians.io/profilers/event_prof) (as `:event_prof`)\n- [TagProf](https://test-prof.evilmartians.io/profilers/tag_prof) (as `:tag_prof`)\n- [StackProf](https://test-prof.evilmartians.io/profilers/stack_prof) (as `:stack_prof`)\n- [FactoryProf](https://test-prof.evilmartians.io/profilers/factory_prof) (as `:factory_prof`)\n- [FactoryDefault profiler](https://test-prof.evilmartians.io/recipes/factory_default) (as `:factory_default_prof`)\n\n## Installation\n\nAdd the gem to your project:\n\n```ruby\n# Gemfile\ngroup :development, :test do\n  gem \"test-prof-autopilot\"\nend\n```\n\nMake sure `test-prof-autopilot` is required in your test environment.\n\nThat's it!\n\n[TestProf]: https://test-prof.evilmartians.io/\n","funding_links":["https://github.com/sponsors/test-prof"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftest-prof%2Ftest-prof-autopilot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftest-prof%2Ftest-prof-autopilot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftest-prof%2Ftest-prof-autopilot/lists"}