{"id":13427753,"url":"https://github.com/salsify/goldiloader","last_synced_at":"2025-05-12T15:35:28.728Z","repository":{"id":17851959,"uuid":"20772066","full_name":"salsify/goldiloader","owner":"salsify","description":"Just the right amount of Rails eager loading","archived":false,"fork":false,"pushed_at":"2024-11-08T21:15:24.000Z","size":297,"stargazers_count":1630,"open_issues_count":2,"forks_count":53,"subscribers_count":65,"default_branch":"master","last_synced_at":"2025-04-23T17:37:15.723Z","etag":null,"topics":["activerecord","eager-loading","gem","hacktoberfest","performance","ruby-on-rails"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/salsify.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-06-12T15:13:26.000Z","updated_at":"2025-04-14T14:39:55.000Z","dependencies_parsed_at":"2024-01-31T05:10:06.411Z","dependency_job_id":"25d4c3ab-b2cd-4f74-9994-b453f7b2bb2e","html_url":"https://github.com/salsify/goldiloader","commit_stats":{"total_commits":187,"total_committers":25,"mean_commits":7.48,"dds":0.5775401069518716,"last_synced_commit":"8cdf77c6006e6e045a6fb684376757f2058e692f"},"previous_names":[],"tags_count":41,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salsify%2Fgoldiloader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salsify%2Fgoldiloader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salsify%2Fgoldiloader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salsify%2Fgoldiloader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/salsify","download_url":"https://codeload.github.com/salsify/goldiloader/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253766455,"owners_count":21960921,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["activerecord","eager-loading","gem","hacktoberfest","performance","ruby-on-rails"],"created_at":"2024-07-31T01:00:39.893Z","updated_at":"2025-05-12T15:35:28.698Z","avatar_url":"https://github.com/salsify.png","language":"Ruby","readme":"# Goldiloader\n\n[![Gem Version](https://badge.fury.io/rb/goldiloader.svg)][gem]\n[![Build Status](https://circleci.com/gh/salsify/goldiloader.svg?style=svg)][circleci]\n[![Code Climate](https://codeclimate.com/github/salsify/goldiloader.svg)][codeclimate]\n[![Coverage Status](https://coveralls.io/repos/salsify/goldiloader/badge.svg)][coveralls]\n\n[gem]: https://rubygems.org/gems/goldiloader\n[circleci]: https://circleci.com/gh/salsify/goldiloader\n[codeclimate]: https://codeclimate.com/github/salsify/goldiloader\n[coveralls]: https://coveralls.io/r/salsify/goldiloader\n\nWouldn't it be awesome if ActiveRecord didn't make you think about eager loading and it just did the \"right\" thing by default? With Goldiloader it can!\n\n**This branch only supports Rails 6.1+ with Ruby 3.0+. For older versions of Rails/Ruby use\n[release-4.x](https://github.com/salsify/goldiloader/blob/release-4.x/README.md),\n[release-3.x](https://github.com/salsify/goldiloader/blob/release-3.x/README.md),\n[release-2.x](https://github.com/salsify/goldiloader/blob/release-2.x/README.md)\nor [release-1.x](https://github.com/salsify/goldiloader/blob/release-1.x/README.md).**\n\nConsider the following models:\n\n```ruby\nclass Blog \u003c ActiveRecord::Base\n  has_many :posts\nend\n\nclass Post \u003c ActiveRecord::Base\n  belongs_to :blog\nend\n```\n\nHere are some sample queries without the Goldiloader:\n\n```ruby\n\u003e blogs = Blog.limit(5).to_a\n# SELECT * FROM blogs LIMIT 5\n\n\u003e blogs.each { |blog| blog.posts.to_a }\n# SELECT * FROM posts WHERE blog_id = 1\n# SELECT * FROM posts WHERE blog_id = 2\n# SELECT * FROM posts WHERE blog_id = 3\n# SELECT * FROM posts WHERE blog_id = 4\n# SELECT * FROM posts WHERE blog_id = 5\n```\n\nHere are the same queries with the Goldiloader:\n\n```ruby\n\u003e blogs = Blog.limit(5).to_a\n# SELECT * FROM blogs LIMIT 5\n\n\u003e blogs.each { |blog| blog.posts.to_a }\n# SELECT * FROM posts WHERE blog_id IN (1,2,3,4,5)\n```\n\nWhoa! It automatically loaded all of the posts for our five blogs in a single database query without specifying any eager loads! Goldiloader assumes that you'll access all models loaded from a query in a uniform way. The first time you traverse an association on any of the models it will eager load the association for all the models. It even works with arbitrary nesting of associations.\n\nRead more about the motivation for the Goldiloader in this [blog post](https://www.salsify.com/blog/engineering/automatic-eager-loading-rails).\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n    gem 'goldiloader'\n\nAnd then execute:\n\n    $ bundle\n\nOr install it yourself as:\n\n    $ gem install goldiloader\n\n## Usage\n\nBy default all associations will be automatically eager loaded when they are first accessed so hopefully most use cases should require no additional configuration. Note you're still free to explicitly eager load associations via `eager_load`, `includes`, or `preload`.\n\n### Disabling Automatic Eager Loading\n\nYou can disable automatic eager loading with `auto_include` query scope method:\n\n```ruby\nBlog.order(:name).auto_include(false)\n```\n\nNote this will not disable automatic eager loading for nested associations.\n\nAutomatic eager loading can be disabled for specific associations by customizing the association's scope:\n\n```ruby\nclass Blog \u003c ActiveRecord::Base\n  has_many :posts, -\u003e { auto_include(false) }\nend\n```\n\nAutomatic eager loading can be disabled globally disabled for all threads:\n\n```ruby\n# config/initializers/goldiloader.rb\nGoldiloader.globally_enabled = false\n```\n\nAutomatic eager loading can then be selectively enabled for particular sections of code:\n\n```ruby\n# Using a block form\nGoldiloader.enabled do\n  # Automatic eager loading is enabled for the current thread\n  # ...\nend\n\n# Using a non-block form\nGoldiloader.enabled = true\n# Automatic eager loading is enabled for the current thread\n# ...\nGoldiloader.enabled = false\n```\n\nSimilarly, you can selectively disable automatic eager loading for particular sections of code in a thread local manner:\n\n```ruby\n# Using a block form\nGoldiloader.disabled do\n  # Automatic eager loading is disabled for the current thread\n  # ...\nend\n\n# Using a non-block form\nGoldiloader.enabled = false\n# Automatic eager loading is disabled for the current thread\n# ...\nGoldiloader.enabled = true\n```\n\nNote `Goldiloader.enabled=`, `Goldiloader.enabled`, and `Goldiloader.disabled` are thread local to ensure\nproper thread isolation in multi-threaded servers like Puma.\n\n### Association Options\n\nGoldiloader supports a few options on ActiveRecord associations to customize its behavior.\n\n#### fully_load\n\nThere are several association methods that ActiveRecord can either execute on in memory models or push down into SQL depending on whether or not the association is loaded. This includes the following methods:\n\n* `first`\n* `second`\n* `third`\n* `fourth`\n* `fifth`\n* `forty_two` (one of the hidden gems in Rails 4.1)\n* `last`\n* `size`\n* `ids_reader`\n* `empty?`\n* `exists?`\n\nThis can cause problems for certain usage patterns if we're no longer specifying eager loads:\n\n```ruby\n\u003e blogs = Blog.limit(5).to_a\n# SELECT * FROM blogs LIMIT 5\n\n\u003e blogs.each do |blog|\n    if blog.posts.exists?\n      puts blog.posts\n    else\n      puts 'No posts'\n  end\n# SELECT 1 AS one FROM posts WHERE blog_id = 1 LIMIT 1\n# SELECT * FROM posts WHERE blog_id IN (1,2,3,4,5)\n```\n\nNotice the first call to `blog.posts.exists?` was executed via SQL because the `posts` association wasn't yet loaded. The `fully_load` option can be used to force ActiveRecord to fully load the association (and do any necessary automatic eager loading) when evaluating methods like `exists?`:\n\n```ruby\nclass Blog \u003c ActiveRecord::Base\n  has_many :posts, fully_load: true\nend\n```\n\n## Limitations\n\nGoldiloader leverages the ActiveRecord eager loader so it shares some of the same limitations. See [eager loading workarounds](#eager-loading-limitation-workarounds) for some potential workarounds.\n\n### has_one associations that rely on a SQL limit\n\nYou should not try to auto eager load (or regular eager load) `has_one` associations that actually correspond to multiple records and rely on a SQL limit to only return one record. Consider the following example:\n\n```ruby\nclass Blog \u003c ActiveRecord::Base\n  has_many :posts\n  has_one :most_recent_post, -\u003e { order(published_at: desc) }, class_name: 'Post'\nend\n```\n\nWith standard Rails lazy loading the `most_recent_post` association is loaded with a query like this:\n\n```sql\nSELECT * FROM posts WHERE blog_id = 1 ORDER BY published_at DESC LIMIT 1\n```\n\nWith auto eager loading (or regular eager loading) the `most_recent_post` association is loaded with a query like this:\n\n```sql\nSELECT * FROM posts WHERE blog_id IN (1,2,3,4,5) ORDER BY published_at DESC\n```\n\nNotice the SQL limit can no longer be used which results in fetching all posts for each blog. This can cause severe performance problems if there are a large number of posts.\n\n### Other Limitations\n\nAssociations with any of the following options cannot be eager loaded:\n\n* `limit`\n* `offset`\n* `finder_sql`\n\nGoldiloader detects associations with any of these options and disables automatic eager loading on them.\n\nIt might still be possible to eager load these with Goldiloader by using [custom preloads](#custom-preloads).\n\n\n### Eager Loading Limitation Workarounds\n\nMost of the Rails limitations with eager loading can be worked around by pushing the problematic SQL into the database via lateral joins (or database views if your database doesn't support lateral joins). Consider the following example with associations that can't be eager loaded due to SQL limits:\n\n```ruby\nclass Blog \u003c ActiveRecord::Base\n  has_many :posts\n  has_one :most_recent_post, -\u003e { order(published_at: desc) }, class_name: 'Post'\n  has_many :recent_posts, -\u003e { order(published_at: desc).limit(5) }, class_name: 'Post'\nend\n```\nThis can be reworked to push the order/limit into lateral joins like this:\n\n```ruby\nclass Blog \u003c ActiveRecord::Base\n  has_many :posts\n  has_one :most_recent_post, -\u003e {\n    joins(Arel.sql(\u003c\u003c-SQL.squish))\n      INNER JOIN LATERAL (\n        SELECT id\n        FROM posts p1\n        WHERE blog_id = posts.blog_id\n        ORDER BY published_at DESC\n        LIMIT 1\n      ) p2 on (p2.id = posts.id)\n    SQL\n  }, class_name: 'Post'\n  has_many :recent_posts, -\u003e {\n    joins(Arel.sql(\u003c\u003c-SQL.squish))\n      INNER JOIN LATERAL (\n        SELECT id\n        FROM posts p1\n        WHERE blog_id = posts.blog_id\n        ORDER BY published_at DESC\n        LIMIT 5\n      ) p2 on (p2.id = posts.id)\n    SQL\n  }, class_name: 'Post'\nend\n```\n\n## Custom Preloads\n\nIn addition to preloading relations, you can also define custom preloads by yourself in your model. The only requirement is that you need to be able to perform a lookup for multiple records/ids and return a single Hash with the ids as keys.\nIf that's the case, these preloads can nearly be anything. Some examples could be:\n\n- simple aggregations (count, sum, maximum, etc.)\n- more complex custom SQL queries\n- external API requests (ElasticSearch, Redis, etc.)\n- relations with primary keys stored in a `jsonb` column\n\nHere's how:\n\n```ruby\nclass Blog \u003c ActiveRecord::Base\n  has_many :posts\n\n  def posts_count\n    goldiload do |ids|\n      # By default, `ids` will be an array of `Blog#id`s\n      Post\n        .where(blog_id: ids)\n        .group(:blog_id)\n        .count\n    end\n  end\nend\n```\n\nThe first time you call the `posts_count` method, it will call the block with all model ids from the current context and reuse the result from the block for all other models in the context.\n\nA more complex example might use a custom primary key instead of `id`, use a non ActiveRecord API and have more complex return values than just scalar values:\n\n\n```ruby\nclass Post \u003c ActiveRecord::Base\n  def main_translator_reference\n    json_payload[:main_translator_reference]\n  end\n\n  def main_translator\n    goldiload(key: :main_translator_reference) do |references|\n      # `references` will be an array of `Post#main_translator_reference`s\n      SomeExternalApi.fetch_translators(\n        id: references\n      ).index_by(\u0026:id)\n    end\n  end\nend\n```\n\nIf you want to preload something that is based on multiple keys, you can also pass an array:\n\n```ruby\nclass Meeting \u003c ActiveRecord::Base\n  def organizer_notes\n    goldiload(key: [:organizer_id, :room_id]) do |id_sets|\n      # +id_sets+ will be a two dimensional array with the\n      # organizer_id and room_id for each item, e.g.\n      # [\n      #   [\u003corganizer_id_1\u003e, \u003croom_id_1\u003e],\n      #   [\u003corganizer_id_2\u003e, \u003croom_id_2\u003e]\n      # ]\n      notes = logic_for_fetching_organizer_notes\n      notes.group_by do |report|\n        [report.organizer_id, report.room_id]\n      end\n    end\n  end\nend\n```\n\n\n**Note:** The `goldiload` method will use the `source_location` of the given block as a cache name to distinguish between multiple defined preloads. If this causes an issue for you, you can also pass a cache name explicitly as the first argument to the `goldiload` method.\n\n\n### Gotchas\n\nEven though the methods in the examples above (`posts_count`, `main_translator`) are actually instance methods, the block passed to `goldiload` should not contain any references to these instances, as this could break the internal lookup/caching mechanism. We prevent this for the `self` keyword, so you'll get a `NoMethodError`. If you get this, you might want to think about the implementation rather than just trying to work around the exception.\n\n\n## Upgrading\n\n### From 0.x, 1.x\n\nThe `auto_include` association option has been removed in favor of the `auto_include` query scope method.\nAssociations that specify this option must migrate to use the query scope method:\n\n```ruby\nclass Blog \u003c ActiveRecord::Base\n  # Old syntax\n  has_many :posts, auto_include: false\n\n  # New syntax\n  has_many :posts, -\u003e { auto_include(false) }\nend\n```\n\n## Status\n\nThis gem is tested with Rails 6.1, 7.0, 7.1, 7.2, and Edge using MRI 3.0, 3.1, 3.2, and 3.3.\n\nLet us know if you find any issues or have any other feedback.\n\n## Change log\n\nSee the [change log](https://github.com/salsify/goldiloader/blob/master/CHANGELOG.md).\n\n## Contributing\n\n1. Fork it\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create new Pull Request\n","funding_links":[],"categories":["Active Record","ORM/ODM Extensions","Ruby","模型","Gems"],"sub_categories":["Omniauth","Performance Optimization"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalsify%2Fgoldiloader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsalsify%2Fgoldiloader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalsify%2Fgoldiloader/lists"}