{"id":14991259,"url":"https://github.com/wetransfer/wt_activerecord_index_spy","last_synced_at":"2025-04-09T13:06:40.099Z","repository":{"id":47465968,"uuid":"342570621","full_name":"WeTransfer/wt_activerecord_index_spy","owner":"WeTransfer","description":"A gem to spy queries running with Active Record and report missing indexes","archived":false,"fork":false,"pushed_at":"2024-12-11T14:56:51.000Z","size":485,"stargazers_count":78,"open_issues_count":4,"forks_count":2,"subscribers_count":16,"default_branch":"main","last_synced_at":"2025-04-09T13:06:35.831Z","etag":null,"topics":["activerecord","database-indexes","mysql","postgersql","rails","rspec","ruby","wt-branch-protection-default"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WeTransfer.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-26T12:34:00.000Z","updated_at":"2024-12-11T14:56:57.000Z","dependencies_parsed_at":"2024-09-11T22:36:26.319Z","dependency_job_id":"969ad47d-5a1b-427f-a6a3-d6caefe336c1","html_url":"https://github.com/WeTransfer/wt_activerecord_index_spy","commit_stats":{"total_commits":85,"total_committers":4,"mean_commits":21.25,"dds":"0.12941176470588234","last_synced_commit":"f81d4bcec5163119cb908fc64ac3beb602b65637"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeTransfer%2Fwt_activerecord_index_spy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeTransfer%2Fwt_activerecord_index_spy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeTransfer%2Fwt_activerecord_index_spy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeTransfer%2Fwt_activerecord_index_spy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WeTransfer","download_url":"https://codeload.github.com/WeTransfer/wt_activerecord_index_spy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248045231,"owners_count":21038553,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["activerecord","database-indexes","mysql","postgersql","rails","rspec","ruby","wt-branch-protection-default"],"created_at":"2024-09-24T14:22:02.400Z","updated_at":"2025-04-09T13:06:40.083Z","avatar_url":"https://github.com/WeTransfer.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# wt_activerecord_index_spy\n\n[![Gem](https://img.shields.io/gem/v/wt_activerecord_index_spy)](https://rubygems.org/gems/wt_activerecord_index_spy)\n![GitHub Actions Workflow](https://github.com/WeTransfer/wt_activerecord_index_spy/actions/workflows/main.yml/badge.svg)\n[![Hippocratic License](https://img.shields.io/badge/license-Hippocratic-green)](https://github.com/WeTransfer/wt_activerecord_index_spy/blob/main/LICENSE.md)\n\nA Ruby library to watch and analyze queries that run using `ActiveRecord` to check\nif they use a proper index.\n\nIt subscribes to `sql.active_record` notification using `ActiveSupport::Notifications`.\n\nIt was designed to be used in tests, but it's also possible to use it in\nstaging or production, carefully.\n\n## Why would I use this?\n\nImagine you have an application running in production and after a deploy, it starts to slow down.\n\nAfter a perhaps exhaustive debugging session, you may find that a new query or perhaps a change\nin the database schema was responsible for starting to have queries that do not use database indexes.\n\nThen, you create the appropriate index and the problem is solved!\n\nBy using this gem, you can get those queries that are not using suitable database indexes in\nyour test suite. So you won't have surprises like the example above, after a deploy.\n\nYou can also enable the gem in your development/staging environment, generate a report\nand analyze if there is any missing index.\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'wt_activerecord_index_spy'\n```\n\nAnd then execute:\n\n    $ bundle install\n\nOr install it yourself as:\n\n    $ gem install wt_activerecord_index_spy\n\n## Usage\n\nThere are 3 different modes to use it:\n\n### 1 - Using a test matcher\n\nInclude the helper in your RSpec configuration:\n\n```ruby\nrequire 'wt_activerecord_index_spy/test_helpers'\n\nRSpec.configure do |config|\n  config.include(WtActiverecordIndexSpy::TestHelpers)\nend\n```\n\nUse the helper `have_used_db_indexes` where you want to check if all queries used database indexes:\n\n```ruby\nit 'uses an index for all the queries' do\n  expect { SomeClass.some_method }.to have_used_db_indexes\nend\n```\n\nGiven that some results are uncertain, it's also possible to set the matcher to fail only with certain results:\n\n```ruby\nit 'uses an index for all the queries' do\n  expect { SomeClass.some_method }.to have_used_db_indexes(only_certains: true)\nend\n```\n\n#### Run for all rspec tests\n\nBy adding the following to your `rspec` configuration the `have_used_db_indexes` will run on each individual test and error if an index has not been used:\n\n```ruby\nRspec.configure do |config|\n  config.around(:each) do |example|\n    unless example.metadata[:skip_index_spy]\n      expect { example.run }.to(have_used_db_indexes)\n    else\n      example.run\n    end\n  end\n\n  config.after(:all) do\n    WtActiverecordIndexSpy.export_html_results\n  end\nend\n```\n\nIf you wish to skip index checking for specific tests you can then annotate your test as follows:\n\n```ruby\ndescribe 'Will not check indexes', :skip_index_spy do\n# ...or...\ncontext 'Does not check indexes', :skip_index_spy do\n# ...or...\nit 'will not check indexes', :skip_index_spy do\n```\n\n### 2 - Watching all queries from a start point\n\nAdd this line to enable it:\n\n```ruby\nWtActiverecordIndexSpy.watch_queries\n```\n\nAfter that, `wt_activerecord_index_spy` will run an `EXPLAIN` query for every query\nfired with `ActiveRecord` that has a `WHERE` condition.\n\nFinally, you can generate a report with the results:\n\n```ruby\nWtActiverecordIndexSpy.export_html_results\n```\n\nThis method creates an HTML file with a report and prints its location to STDOUT.\n\nThe content of this file is similar to this:\n\n| Level | Identifier | Query | Origin |\n| ----  | ---------- | ----- | ------ |\n| certain | User Load | SELECT `users`.* FROM `users` WHERE `users`.`name` = 'lala' LIMIT 1  | spec/wt_activerecord_index_spy_spec.rb:162 |\n| uncertain | User Load | SELECT `users`.* FROM `users` WHERE `users`.`city_id` IN (SELECT `cities`.`id` FROM `cities` WHERE `cities`.`name` = 'Santo Andre') | spec/wt_activerecord_index_spy_spec.rb:173 |\n\nWhere:\n- **Level**: `certain` when it is certain that an index is missing, or `uncertain` when it's not possible to be sure\n- **Identifier**: is the query identifier reported `ActiveRecord` notification\n- **Origin**: is the line the query was fired\n\nThis mode, by default, **ignores** queries that were originated in test code. For that, it considers files which path includes `test/` or `spec/`.\n\nIt's possible to disable it as follows:\n\n```ruby\nWtActiverecordIndexSpy.watch_queries(ignore_queries_originated_in_test_code: false)\n```\n\nIf the same query runs in many places, only one origin will be added to the report.\n\nIt's also possible to get the results using the following methods:\n\n```ruby\n# Returns a list of certain results\nWtActiverecordIndexSpy.certain_results\n\n# Returns a list of certain and uncertain results mixed\nWtActiverecordIndexSpy.results\n```\n\n### 3 - Watching all queries given a block\n\nIt's also possible to enable it in a specific context, using a block:\n\n```ruby\nWtActiverecordIndexSpy.watch_queries do\n  # some code...\nend\n```\n\nAfter that, you can generate the HTML report with:\n\n```ruby\nWtActiverecordIndexSpy.export_html_results\n```\n\n## Supported versions\n\nCurrently, it supports:\n\n**Ruby**: 3.0, 2.7, 2.6, 2.5\n\n**Mysql**: 5.7\n\n**PostgreSQL**: 13.2\n\nNote: Currently, the PostgreSQL query analyser is not so intelligent and can't be\ncertain if an index is missing or not. So all results are `uncertain`. More\ndetails in https://github.com/WeTransfer/wt_activerecord_index_spy/issues/12\n\n**ActiveRecord**: 6, 5, 4\n\n**RSpec**: 3.x\n\n## Development\n\nAfter checking out the repo, run `cp .env.template .env` and set database credentials to run the tests.\n\nRun `bin/setup` to install dependencies.\n\nThen, run `bundle exec rake db:create db:migrate` to create the Mysql databases and `ADAPTER=postgresql bundle exec rake db:create db:migrate` to create the PostgresSQL database.\n\nThen, run `bundle exec rspec` to run the tests for Mysql and `ADAPTER=postgresql bundle exec rspec`.\n\nYou can also run `bundle exec bin/console` for an interactive prompt that will allow you to experiment.\n\nTo install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/wetransfer/wt_activerecord_index_spy. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](./CODE_OF_CONDUCT.md).\nPlease add your name to the [CONTRIBUTORS.md](./CONTRIBUTORS.md)\n\n## License\n\nThe gem is available as open source under the terms of the [Hippocratic License](https://firstdonoharm.dev/version/2/1/license.html).\n\n## Code of Conduct\n\nEveryone interacting in the WT Activerecord Index Spy project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](./CODE_OF_CONDUCT.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwetransfer%2Fwt_activerecord_index_spy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwetransfer%2Fwt_activerecord_index_spy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwetransfer%2Fwt_activerecord_index_spy/lists"}