{"id":13878209,"url":"https://github.com/fatkodima/pluck_in_batches","last_synced_at":"2025-04-08T09:09:56.473Z","repository":{"id":165925276,"uuid":"640033084","full_name":"fatkodima/pluck_in_batches","owner":"fatkodima","description":"A faster alternative to the custom use of `in_batches` with `pluck`","archived":false,"fork":false,"pushed_at":"2024-11-03T19:37:21.000Z","size":47,"stargazers_count":152,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-01T08:38:45.788Z","etag":null,"topics":["activerecord","gem","performance","rails","ruby"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fatkodima.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-12T20:14:00.000Z","updated_at":"2025-03-25T14:39:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"7fbd81af-b0f1-48f1-a3ea-ddd55300ea92","html_url":"https://github.com/fatkodima/pluck_in_batches","commit_stats":{"total_commits":22,"total_committers":1,"mean_commits":22.0,"dds":0.0,"last_synced_commit":"c0209d9b1af65cb0f4d2dabc5df82df5c2a7d602"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatkodima%2Fpluck_in_batches","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatkodima%2Fpluck_in_batches/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatkodima%2Fpluck_in_batches/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatkodima%2Fpluck_in_batches/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fatkodima","download_url":"https://codeload.github.com/fatkodima/pluck_in_batches/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247809964,"owners_count":20999816,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["activerecord","gem","performance","rails","ruby"],"created_at":"2024-08-06T08:01:42.785Z","updated_at":"2025-04-08T09:09:56.448Z","avatar_url":"https://github.com/fatkodima.png","language":"Ruby","funding_links":[],"categories":["Ruby","Optimizations"],"sub_categories":[],"readme":"# PluckInBatches\n\n[![Build Status](https://github.com/fatkodima/pluck_in_batches/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/fatkodima/pluck_in_batches/actions/workflows/ci.yml)\n\nActiveRecord comes with `find_each` / `find_in_batches` / `in_batches` methods to batch process records from a database.\nActiveRecord also has the `pluck` method which allows the selection of a set of fields without pulling\nthe entire record into memory.\n\nThis gem combines these ideas and provides `pluck_each` and `pluck_in_batches` methods to allow\nbatch processing of plucked fields from the database.\n\nIt performs half of the number of SQL queries, allocates up to half of the memory and is up to 2x faster\n(or more, depending on how far is your database from the application) than the available alternative:\n\n```ruby\n# Before\nUser.in_batches do |batch| # or .find_in_batches, or .select(:email).find_each etc\n  emails = batch.pluck(:emails)\n  # do something with emails\nend\n\n# Now, using this gem (up to 2x faster)\nUser.pluck_in_batches(:email) do |emails|\n  # do something with emails\nend\n```\n\n**Note**: You may also find [`sidekiq-iteration`](https://github.com/fatkodima/sidekiq-iteration) useful when iterating over large collections in Sidekiq jobs.\n\n## Requirements\n\n- Ruby 2.7+\n- ActiveRecord 6+\n\nIf you need support for older versions, [open an issue](https://github.com/fatkodima/pluck_in_batches/issues/new).\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'pluck_in_batches'\n```\n\nAnd then execute:\n\n```sh\n$ bundle\n```\n\nOr install it yourself as:\n\n```sh\n$ gem install pluck_in_batches\n```\n\n## Usage\n\n### `pluck_each`\n\nBehaves similarly to `find_each` ActiveRecord's method, but yields each set of values corresponding\nto the specified columns.\n\n```ruby\n# Single column\nUser.where(active: true).pluck_each(:email) do |email|\n  # do something with email\nend\n\n# Multiple columns\nUser.where(active: true).pluck_each(:id, :email) do |id, email|\n  # do something with id and email\nend\n```\n\n### `pluck_in_batches`\n\nBehaves similarly to `in_batches` ActiveRecord's method, but yields each batch\nof values corresponding to the specified columns.\n\n```ruby\n# Single column\nUser.where(\"age \u003e 21\").pluck_in_batches(:email) do |emails|\n  jobs = emails.map { |email| PartyReminderJob.new(email) }\n  ActiveJob.perform_all_later(jobs)\nend\n\n# Multiple columns\nUser.pluck_in_batches(:name, :email).with_index do |group, index|\n  puts \"Processing group ##{index}\"\n  jobs = group.map { |name, email| PartyReminderJob.new(name, email) }\n  ActiveJob.perform_all_later(jobs)\nend\n\n# Custom arel column\nUser.pluck_in_batches(:id, Arel.sql(\"json_extract(users.metadata, '$.rank')\")).with_index do |group, index|\n  # ...\nend\n```\n\nBoth methods support the following configuration options:\n\n* `:batch_size` - Specifies the size of the batch. Defaults to 1000.\n  Also aliased as `:of`.\n* `:start` - Specifies the primary key value to start from, inclusive of the value.\n* `:finish` - Specifies the primary key value to end at, inclusive of the value.\n* `:error_on_ignore` - Overrides the application config to specify if an error should be raised when\n  an order is present in the relation.\n* `:cursor_column` - Specifies the column(s) on which the iteration should be done.\n  This column(s) should be orderable (e.g. an integer or string). Defaults to primary key.\n* `:order` - Specifies the primary key order (can be `:asc` or `:desc` or\n  an array consisting of :asc or :desc). Defaults to `:asc`.\n\n## Development\n\nTo install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/fatkodima/pluck_in_batches.\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffatkodima%2Fpluck_in_batches","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffatkodima%2Fpluck_in_batches","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffatkodima%2Fpluck_in_batches/lists"}