{"id":13878220,"url":"https://github.com/fatkodima/fast_count","last_synced_at":"2025-04-04T02:07:49.130Z","repository":{"id":156236038,"uuid":"631370224","full_name":"fatkodima/fast_count","owner":"fatkodima","description":"Quickly get a count estimation for large tables (\u003e99% of accuracy for PostgreSQL).","archived":false,"fork":false,"pushed_at":"2024-08-12T13:59:19.000Z","size":39,"stargazers_count":225,"open_issues_count":0,"forks_count":7,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-10-30T05:57:25.578Z","etag":null,"topics":["gem","mysql","performance","postgresql","rails","ruby"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fatkodima.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-22T20:07:44.000Z","updated_at":"2024-10-25T17:57:49.000Z","dependencies_parsed_at":null,"dependency_job_id":"a10baca9-00d3-4372-90b2-9fdb05a367ce","html_url":"https://github.com/fatkodima/fast_count","commit_stats":{"total_commits":24,"total_committers":2,"mean_commits":12.0,"dds":0.04166666666666663,"last_synced_commit":"90da58f350d2e7bb05dfed17f4a4a8524479db82"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatkodima%2Ffast_count","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatkodima%2Ffast_count/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatkodima%2Ffast_count/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatkodima%2Ffast_count/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fatkodima","download_url":"https://codeload.github.com/fatkodima/fast_count/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247107820,"owners_count":20884795,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gem","mysql","performance","postgresql","rails","ruby"],"created_at":"2024-08-06T08:01:43.104Z","updated_at":"2025-04-04T02:07:49.113Z","avatar_url":"https://github.com/fatkodima.png","language":"Ruby","readme":"# FastCount\n\n[![Build Status](https://github.com/fatkodima/fast_count/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/fatkodima/fast_count/actions/workflows/ci.yml)\n\nUnfortunately, it's currently notoriously difficult and expensive to get an exact count on large tables.\n\nLuckily, there are [some tricks](https://www.citusdata.com/blog/2016/10/12/count-performance) for quickly getting fairly accurate estimates. For example, on a PostgreSQL table with over 450 million records, you can get a 99.82% accurate count within a fraction of the time. See the table below for an example dataset.\n\n| SQL | Result | Accuracy | Time |\n| --- | --- | --- | --- |\n| `SELECT count(*) FROM small_table` | `2037104` | `100.000%` | `4.900s` |\n| `SELECT fast_count('small_table')` | `2036407` | `99.965%` | `0.050s` |\n| `SELECT count(*) FROM medium_table` | `81716243` | `100.000%` | `257.5s` |\n| `SELECT fast_count('medium_table')` | `81600513` | `99.858%` | `0.048s` |\n| `SELECT count(*) FROM large_table` | `455270802` | `100.000%` | `310.6s` |\n| `SELECT fast_count('large_table')` | `454448393` | `99.819%` | `0.046s` |\n\n*These metrics were pulled from real PostgreSQL databases being used in a production environment.*\n\nFor MySQL, this gem uses internal statistics to return the estimated table's size. And as [per documentation](https://dev.mysql.com/doc/refman/8.0/en/show-table-status.html), it may vary from the actual value by as much as 40% to 50%.\nBut still is useful to get a rough idea of the number of rows in very large tables (where `COUNT(*)` can literally take hours).\n\nSupports PostgreSQL, MySQL, MariaDB, and SQLite.\n\n## Requirements\n\n- Ruby 2.7+\n- ActiveRecord 6+\n\nIf you need support for older versions, [open an issue](https://github.com/fatkodima/fast_count/issues/new).\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'fast_count'\n```\n\nAnd then execute:\n\n```sh\n$ bundle\n```\n\nOr install it yourself as:\n\n```sh\n$ gem install fast_count\n```\n\nIf you are using PostgreSQL, you need to create a database function, used internally:\n\n```sh\n$ rails generate migration install_fast_count\n```\n\nwith the content:\n\n```ruby\nclass InstallFastCount \u003c ActiveRecord::Migration[7.0]\n  def up\n    FastCount.install\n  end\n\n  def down\n    FastCount.uninstall\n  end\nend\n```\n\n## Usage\n\n### Estimated table count\n\nTo quickly get an estimated count of the rows in a table:\n\n```ruby\nUser.fast_count # =\u003e 1_254_312_219\n```\n\n### Result set size estimation\n\nIf you want to quickly get an estimation of how many rows will the query return, without actually executing it, yo can run:\n\n```ruby\nUser.where.missing(:avatar).estimated_count # =\u003e 324_200\n```\n\n**Note**: `estimated_count` relies on the database query planner estimations (basically on the output of `EXPLAIN`) to get its results and can be very imprecise. It is better be used to get an idea of the order of magnitude of the future result.\n\n### Exact distinct values count\n\nTo quickly get an exact number of distinct values in a column, you can run:\n\n```ruby\nUser.fast_distinct_count(column: :company_id) # =\u003e 243\n```\n\nIt is suited for cases when there is a small amount of distinct values in a column compared to a total number\nof values (for example, 10M rows total and 200 distinct values).\n\nRuns orders of magnitude faster than `SELECT COUNT(DISTINCT column) FROM table`.\n\n**Note**: You need to have an index starting with the specified column for this to work.\n\nUses a [\"Loose Index Scan\" technique](https://wiki.postgresql.org/wiki/Loose_indexscan).\n\n## Configuration\n\nYou can override the following default options:\n\n```ruby\n# Determines for how large tables this gem should get the exact row count using SELECT COUNT.\n# If the approximate row count is smaller than this value, SELECT COUNT will be used,\n# otherwise the approximate count will be used.\nFastCount.threshold = 100_000\n```\n\n## Credits\n\nThanks to [quick_count gem](https://github.com/TwilightCoders/quick_count) for the original idea.\n\n## Development\n\nTo install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/fatkodima/fast_count.\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n","funding_links":[],"categories":["Ruby","Optimizations","Gems"],"sub_categories":["Performance Optimization"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffatkodima%2Ffast_count","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffatkodima%2Ffast_count","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffatkodima%2Ffast_count/lists"}