{"id":26197293,"url":"https://github.com/mongoid/mongoid-scroll","last_synced_at":"2025-04-06T13:11:49.672Z","repository":{"id":6953181,"uuid":"8205116","full_name":"mongoid/mongoid-scroll","owner":"mongoid","description":"Mongoid extension that enables infinite scrolling with MongoDB.","archived":false,"fork":false,"pushed_at":"2024-09-07T14:02:53.000Z","size":198,"stargazers_count":60,"open_issues_count":2,"forks_count":12,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-03-30T11:09:54.481Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mongoid.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2013-02-14T18:34:04.000Z","updated_at":"2024-09-07T14:02:57.000Z","dependencies_parsed_at":"2024-06-19T01:30:01.453Z","dependency_job_id":"a7538a67-0fea-4e5b-9deb-968d77323516","html_url":"https://github.com/mongoid/mongoid-scroll","commit_stats":{"total_commits":112,"total_committers":9,"mean_commits":"12.444444444444445","dds":0.4017857142857143,"last_synced_commit":"ddedd8c5761759680092e4bde178a4aafdd36d68"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mongoid%2Fmongoid-scroll","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mongoid%2Fmongoid-scroll/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mongoid%2Fmongoid-scroll/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mongoid%2Fmongoid-scroll/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mongoid","download_url":"https://codeload.github.com/mongoid/mongoid-scroll/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247485290,"owners_count":20946398,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-12T02:25:30.038Z","updated_at":"2025-04-06T13:11:49.646Z","avatar_url":"https://github.com/mongoid.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"- [Mongoid::Scroll](#mongoidscroll)\n  - [Compatibility](#compatibility)\n  - [Demo](#demo)\n  - [The Problem](#the-problem)\n  - [Installation](#installation)\n  - [Usage](#usage)\n  - [Indexes and Performance](#indexes-and-performance)\n  - [Cursors](#cursors)\n    - [Standard Cursor](#standard-cursor)\n    - [Base64 Encoded Cursor](#base64-encoded-cursor)\n  - [Contributing](#contributing)\n  - [Copyright and License](#copyright-and-license)\n\n# Mongoid::Scroll\n\n[![Gem Version](https://badge.fury.io/rb/mongoid-scroll.svg)](https://badge.fury.io/rb/mongoid-scroll)\n[![Build Status](https://github.com/mongoid/mongoid-scroll/actions/workflows/ci.yml/badge.svg)](https://github.com/mongoid/mongoid-scroll/actions/workflows/ci.yml)\n[![Coverage Status](https://coveralls.io/repos/github/mongoid/mongoid-scroll/badge.svg?branch=master)](https://coveralls.io/github/mongoid/mongoid-scroll?branch=master)\n[![Code Climate](https://codeclimate.com/github/mongoid/mongoid-scroll.svg)](https://codeclimate.com/github/mongoid/mongoid-scroll)\n\nMongoid extension that enables infinite scrolling for `Mongoid::Criteria` and `Mongo::Collection::View`.\n\n## Compatibility\n\nThis gem supports Mongoid 6, 7, 8 and 9.\n\n## Demo\n\nTake a look at [this example](examples/feed.rb). Try with with `bundle exec ruby examples/feed.rb`.\n\n## The Problem\n\nTraditional pagination does not work when data changes between paginated requests, which makes it unsuitable for infinite scroll behaviors.\n\n* If a record is inserted before the current page limit, items will shift right, and the next page will include a duplicate.\n* If a record is removed before the current page limit, items will shift left, and the next page will be missing a record.\n\nThe solution implemented by the `scroll` extension paginates data using a cursor, giving you the ability to restart pagination where you left it off. This is a non-trivial problem when combined with sorting over non-unique record fields, such as timestamps.\n\n## Installation\n\nAdd the gem to your Gemfile and run `bundle install`.\n\n```ruby\ngem 'mongoid-scroll'\n```\n\n## Usage\n\nA sample model.\n\n```ruby\nmodule Feed\n  class Item\n    include Mongoid::Document\n\n    field :title, type: String\n    field :position, type: Integer\n\n    index({ position: 1, _id: 1 })\n  end\nend\n```\n\nScroll by `:position` and save a cursor to the last item.\n\n```ruby\nsaved_iterator = nil\n\nFeed::Item.desc(:position).limit(5).scroll do |record, iterator|\n  # each record, one-by-one\n  saved_iterator = iterator\nend\n```\n\nResume iterating using the saved cursor and save the cursor to go backwards.\n\n```ruby\nFeed::Item.desc(:position).limit(5).scroll(saved_iterator.next_cursor) do |record, iterator|\n  # each record, one-by-one\n  saved_iterator = iterator\nend\n```\n\nLoop over the first records again.\n\n```ruby\nFeed::Item.desc(:position).limit(5).scroll(saved_iterator.previous_cursor) do |record, iterator|\n  # each record, one-by-one\n  saved_iterator = iterator\nend\n```\n\nUse `saved_iterator.first_cursor` to loop over the first records or `saved_iterator.current_cursor` to loop over the same records again.\n\nThe iteration finishes when no more records are available. You can also finish iterating over the remaining records by omitting the query limit.\n\n```ruby\nFeed::Item.desc(:position).limit(5).scroll(saved_iterator.next_cursor) do |record, iterator|\n  # each record, one-by-one\n  saved_iterator = iterator\nend\n```\n\n## Indexes and Performance\n\nA query without a cursor is identical to a query without a scroll.\n\n``` ruby\n# db.feed_items.find().sort({ position: 1 }).limit(7)\nFeed::Item.desc(:position).limit(7).scroll\n```\n\nSubsequent queries use an `$or` to avoid skipping items with the same value as the one at the current cursor position.\n\n``` ruby\n# db.feed_items.find({ \"$or\" : [\n#   { \"position\" : { \"$gt\" : 13 }},\n#   { \"position\" : 13, \"_id\": { \"$gt\" : ObjectId(\"511d7c7c3b5552c92400000e\") }}\n# ]}).sort({ position: 1 }).limit(7)\nFeed:Item.desc(:position).limit(7).scroll(cursor)\n```\n\nThis means you need to hit an index on `position` and `_id`.\n\n``` ruby\n# db.feed_items.ensureIndex({ position: 1, _id: 1 })\n\nmodule Feed\n  class Item\n    ...\n    index({ position: 1, _id: 1 })\n  end\nend\n```\n\n## Cursors\n\nYou can use `Mongoid::Scroll::Cursor.from_record` to generate a cursor. A cursor points at the last record of the iteration and unlike MongoDB cursors will not expire.\n\n```ruby\nrecord = Feed::Item.desc(:position).limit(3).last\ncursor = Mongoid::Scroll::Cursor.from_record(record, { field: Feed::Item.fields[\"position\"] })\n# cursor or cursor.to_s can be returned to a client and passed into .scroll(cursor)\n```\n\nYou can also a `field_name` and `field_type` instead of a Mongoid field.\n\n```ruby\ncursor = Mongoid::Scroll::Cursor.from_record(record, { field_type: DateTime, field_name: \"position\" })\n```\n\nWhen the `include_current` option is set to `true`, the cursor will include the record it points to:\n\n```ruby\nrecord = Feed::Item.desc(:position).limit(3).last\ncursor = Mongoid::Scroll::Cursor.from_record(record, { field: Feed::Item.fields[\"position\"], include_current: true })\nFeed::Item.asc(:position).limit(1).scroll(cursor).first # record\n```\n\nIf the `field_name`, `field_type` or `direction` options you specify when creating the cursor are different from the original criteria, a `Mongoid::Scroll::Errors::MismatchedSortFieldsError` will be raised.\n\n```ruby\ncursor = Mongoid::Scroll::Cursor.from_record(record, { field_type: DateTime, field_name: \"position\" })\nFeed::Item.desc(:created_at).scroll(cursor) # Raises a Mongoid::Scroll::Errors::MismatchedSortFieldsError\n```\n\n### Standard Cursor\n\nThe `Mongoid::Scroll::Cursor` encodes a value and a tiebreak ID separated by `:`, and does not include other options, such as scroll direction. Take extra care not to pass a cursor into a scroll with different options.\n\n### Base64 Encoded Cursor\n\nThe `Mongoid::Scroll::Base64EncodedCursor` can be used instead of `Mongoid::Scroll::Cursor` to generate a base64-encoded string (using RFC 4648) containing all the information needed to rebuild a cursor.\n\n```ruby\nFeed::Item.desc(:position).limit(5).scroll(Mongoid::Scroll::Base64EncodedCursor) do |record, iterator|\n   # iterator.next_cursor is of type Mongoid::Scroll::Base64EncodedCursor\nend\n```\n\n## Contributing\n\nFork the project. Make your feature addition or bug fix with tests. Send a pull request. Bonus points for topic branches.\n\n## Copyright and License\n\nMIT License, see [LICENSE](http://github.com/mongoid/mongoid-scroll/raw/master/LICENSE.md) for details.\n\n(c) 2013-2024 [Daniel Doubrovkine](http://github.com/dblock), based on code by [Frank Macreery](http://github.com/macreery), [Artsy Inc.](http://artsy.net)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmongoid%2Fmongoid-scroll","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmongoid%2Fmongoid-scroll","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmongoid%2Fmongoid-scroll/lists"}