{"id":20471233,"url":"https://github.com/tawk/mongo-cursor-pagination","last_synced_at":"2025-07-24T16:21:22.729Z","repository":{"id":152383133,"uuid":"625886244","full_name":"tawk/mongo-cursor-pagination","owner":"tawk","description":"Cursor based pagination module for native MongoDB driver","archived":false,"fork":false,"pushed_at":"2025-06-17T04:38:29.000Z","size":2178,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-07-09T08:38:25.652Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tawk.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-10T10:22:21.000Z","updated_at":"2023-04-10T10:23:02.000Z","dependencies_parsed_at":"2024-10-12T07:00:34.240Z","dependency_job_id":"576c6c9e-da20-40a8-8232-bf60fc871c21","html_url":"https://github.com/tawk/mongo-cursor-pagination","commit_stats":{"total_commits":186,"total_committers":30,"mean_commits":6.2,"dds":0.7150537634408602,"last_synced_commit":"d8144033764a6b7fb07e23d640e0fcd99dd51feb"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tawk/mongo-cursor-pagination","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tawk%2Fmongo-cursor-pagination","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tawk%2Fmongo-cursor-pagination/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tawk%2Fmongo-cursor-pagination/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tawk%2Fmongo-cursor-pagination/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tawk","download_url":"https://codeload.github.com/tawk/mongo-cursor-pagination/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tawk%2Fmongo-cursor-pagination/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266870562,"owners_count":23998250,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-24T02:00:09.469Z","response_time":99,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T14:15:30.752Z","updated_at":"2025-07-24T16:21:22.681Z","avatar_url":"https://github.com/tawk.png","language":"JavaScript","readme":"# @tawk.to/mongo-cursor-pagination\n\nThis module aids in implementing \"cursor-based\" pagination using Mongo range queries or relevancy-based search results.\n\n## Background\n\nSee this [blog post](https://mixmax.com/blog/api-paging-built-the-right-way) for background on why this library was built.\n\nAPI Pagination is typically implemented one of two different ways:\n\n1. Offset-based paging. This is traditional paging where `skip` and `limit` parameters are passed on the url (or some variation such as `page_num` and `count`). The API would return the results and some indication of whether there is a next page, such as `has_more` on the response. An issue with this approach is that it assumes a static data set; if collection changes while querying, then results in pages will shift and the response will be wrong.\n\n2. Cursor-based paging. An improved way of paging where an API passes back a \"cursor\" (an opaque string) to tell the caller where to query the next or previous pages. The cursor is usually passed using query parameters `next` and `previous`. It's implementation is typically more performant that skip/limit because it can jump to any page without traversing all the records. It also handles records being added or removed because it doesn't use fixed offsets.\n\nThis module helps in implementing #2 - cursor based paging - by providing a method that make it easy to query within a Mongo collection. It also helps by returning a url-safe string that you can return with your HTTP response (see example below).\n\nHere are some examples of cursor-based APIs:\n\n- [Twitter](https://dev.twitter.com/overview/api/cursoring)\n- [Stripe](https://stripe.com/docs/api#pagination-starting_after)\n- [Facebook](https://developers.facebook.com/docs/graph-api/using-graph-api/#cursors)\n\n## Install\n\n`npm install @tawk.to/mongo-cursor-pagination --save`\n\n## Usage\n\n### find()\n\nFind will return ordered and paged results based on a field (`paginatedField`) that you pass in.\n\nCall `find()` with the following parameters:\n\n```\n   Performs a find() query on a passed-in Mongo collection, using criteria you specify. The results\n   are ordered by the paginatedField.\n\n   @param {MongoCollection} collection A collection object returned from the MongoDB library's\n      or the mongoist package's `db.collection(\u003ccollectionName\u003e)` method.\n   @param {Object} params\n      -query {Object} The find query.\n      -limit {Number} The page size. Must be between 1 and `config.MAX_LIMIT`.\n      -fields {Object} Fields to query in the Mongo object format, e.g. {_id: 1, timestamp :1}.\n        The default is to query all fields.\n      -paginatedField {String} The field name to query the range for. The field must be:\n          1. Orderable. We must sort by this value. If duplicate values for paginatedField field\n            exist, the results will be secondarily ordered by the _id.\n          2. Indexed. For large collections, this should be indexed for query performance.\n          3. Immutable. If the value changes between paged queries, it could appear twice.\n          4. Consistent. All values (except undefined and null values) must be of the same type.\n        The default is to use the Mongo built-in '_id' field, which satisfies the above criteria.\n        The only reason to NOT use the Mongo _id field is if you chose to implement your own ids.\n      -sortAscending {Boolean} True to sort using paginatedField ascending (default is false - descending).\n      -sortCaseInsensitive {boolean} Whether to ignore case when sorting, in which case `paginatedField`\n        must be a string property.\n      -next {String} The value to start querying the page.\n      -previous {String} The value to start querying previous page.\n   @param {Function} done Node errback style function.\n```\n\nExample:\n\n```js\nconst mongoist = require('mongoist');\nconst MongoPaging = require('mongo-cursor-pagination');\n\nconst db = mongoist('mongodb://localhost:27017/mydb');\n\nasync function findExample() {\n  await db.collection('myobjects').insertMany([\n    {\n      counter: 1,\n    },\n    {\n      counter: 2,\n    },\n    {\n      counter: 3,\n    },\n    {\n      counter: 4,\n    },\n  ]);\n\n  // Query the first page.\n  let result = await MongoPaging.find(db.collection('myobjects'), {\n    limit: 2,\n  });\n  console.log(result);\n\n  // Query next page.\n  result = await MongoPaging.find(db.collection('myobjects'), {\n    limit: 2,\n    next: result.next, // This queries the next page\n  });\n  console.log(result);\n}\n\nfindExample().catch(console.log);\n```\n\nOutput:\n\n```sh\npage 1 { results:\n   [ { _id: 580fd16aca2a6b271562d8bb, counter: 4 },\n     { _id: 580fd16aca2a6b271562d8ba, counter: 3 } ],\n  next: 'eyIkb2lkIjoiNTgwZmQxNmFjYTJhNmIyNzE1NjJkOGJhIn0',\n  hasNext: true }\npage 2 { results:\n   [ { _id: 580fd16aca2a6b271562d8b9, counter: 2 },\n     { _id: 580fd16aca2a6b271562d8b8, counter: 1 } ],\n  previous: 'eyIkb2lkIjoiNTgwZmQxNmFjYTJhNmIyNzE1NjJkOGI5In0',\n  next: 'eyIkb2lkIjoiNTgwZmQxNmFjYTJhNmIyNzE1NjJkOGI4In0',\n  hasNext: false }\n```\n\n## With Mongoose\n\nInitialize Your Schema\n\n```js\nconst MongoPaging = require('mongo-cursor-pagination');\nconst mongoose = require('mongoose');\nconst counterSchema = new mongoose.Schema({ counter: Number });\n```\n\nPlug the `mongoosePlugin`.\n\n```js\n// this will add paginate function.\ncounterSchema.plugin(MongoPaging.mongoosePlugin);\n\nconst counter = mongoose.model('counter', counterSchema);\n\n// default function is \"paginate\"\ncounter.paginate({ limit: 10 }).then((result) =\u003e {\n  console.log(result);\n});\n```\n\nfor paginate params [refer the find section](https://github.com/mixmaxhq/mongo-cursor-pagination#find)\n\n```js\nconst MongoPaging = require('mongo-cursor-pagination');\nconst mongoose = require('mongoose');\n\nconst counterSchema = new mongoose.Schema({ counter: Number });\n\n// give custom function name\n\ncounterSchema.plugin(MongoPaging.mongoosePlugin, { name: 'paginateFN' });\n\nconst counter = mongoose.model('counter',\ncounterSchema);\n\n// now you can call the custom named function\n\ncounter.paginateFN(params)\n.then(....)\n.catch(....);\n\n```\n\nYou can also use the search function (as described below) like so;\n\n```js\n// this will add paginate function.\ncounterSchema.plugin(MongoPaging.mongoosePlugin);\n\n// give custom function name\n// counterSchema.plugin(MongoPaging.mongoosePlugin, { searchFnName: 'searchFN' });\n\nconst counter = mongoose.model('counter', counterSchema);\n\n// default function is \"paginate\"\ncounter.search('dog', { limit: 10 }).then((result) =\u003e {\n  console.log(result);\n});\n```\n\n### search()\n\nSearch uses Mongo's [text search](https://docs.mongodb.com/v3.2/text-search/) feature and will return paged results ordered by search relevancy. As such, and unlike `find()`, it does not take a `paginatedField` parameter.\n\n```\n   Performs a search query on a Mongo collection and pages the results. This is different from\n   find() in that the results are ordered by their relevancy, and as such, it does not take\n   a paginatedField parameter. Note that this is less performant than find() because it must\n   perform the full search on each call to this function. Also note that results might change\n\n    @param {MongoCollection} collection A collection object returned from the MongoDB library's\n       or the mongoist package's `db.collection(\u003ccollectionName\u003e)` method. This MUST have a Mongo\n       $text index on it.\n      See https://docs.mongodb.com/manual/core/index-text/.\n   @param {String} searchString String to search on.\n   @param {Object} params\n      -query {Object} The find query.\n      -limit {Number} The page size. Must be between 1 and `config.MAX_LIMIT`.\n      -fields {Object} Fields to query in the Mongo object format, e.g. {title :1}.\n        The default is to query ONLY _id (note this is a difference from `find()`).\n      -next {String} The value to start querying the page. Defaults to start at the beginning of\n        the results.\n```\n\nExample:\n\n```js\nconst mongoist = require('mongoist');\nconst MongoPaging = require('mongo-cursor-pagination');\n\nconst db = mongoist('mongodb://localhost:27017/mydb');\n\nasync function searchExample() {\n  await db.collection('myobjects').ensureIndex({\n    mytext: 'text',\n  });\n\n  await db.collection('myobjects').insertMany([\n    {\n      mytext: 'dogs',\n    },\n    {\n      mytext: 'dogs cats',\n    },\n    {\n      mytext: 'dogs cats pigs',\n    },\n  ]);\n\n  // Query the first page.\n  let result = await MongoPaging.search(db.collection('myobjects'), 'dogs', {\n    fields: {\n      mytext: 1,\n    },\n    limit: 2,\n  });\n  console.log(result);\n\n  // Query next page.\n  result = await MongoPaging.search(db.collection('myobjects'), 'dogs', {\n    limit: 2,\n    next: result.next, // This queries the next page\n  });\n  console.log(result);\n}\n\nsearchExample().catch(console.log);\n```\n\nOutput:\n\n```sh\npage 1  { results:\n   [ { _id: 581668318c11596af22a62de, mytext: 'dogs', score: 1 },\n     { _id: 581668318c11596af22a62df, mytext: 'dogs cats', score: 0.75 } ],\n  next: 'WzAuNzUseyIkb2lkIjoiNTgxNjY4MzE4YzExNTk2YWYyMmE2MmRmIn1d' }\npage 2 { results:\n   [ { _id: 581668318c11596af22a62e0, score: 0.6666666666666666 } ] }\n```\n\n### Use with ExpressJS\n\nA popular use of this module is with Express to implement a basic API. As a convenience for this use-case, this library exposes a `findWithReq` function that takes the request object from your Express middleware and returns results:\n\nSo this code using `find()`:\n\n```js\nrouter.get('/myobjects', async (req, res, next) =\u003e {\n  try {\n    const result = await MongoPaging.find(db.collection('myobjects'), {\n      query: {\n        userId: req.user._id\n      },\n      paginatedField: 'created',\n      fields: { // Also need to read req.query.fields to use to filter these fields\n        _id: 1,\n        created: 1\n      },\n      limit: req.query.limit, // Also need to cap this to 25\n      next: req.query.next,\n      previous: req.query.previous,\n    }\n    res.json(result);\n  } catch (err) {\n    next(err);\n  }\n});\n```\n\nIs more elegant with `findWithReq()`:\n\n```js\nrouter.get('/myobjects', async (req, res, next) =\u003e {\n  try {\n    const result = await MongoPaging.findWithReq(req, db.collection('myobjects'), {\n      query: {\n        userId: req.user._id\n      },\n      paginatedField: 'created',\n      fields: {\n        _id: 1,\n        created: 1\n      },\n      limit: 25 // Upper limit\n    }\n    res.json(result);\n  } catch (err) {\n    next(err);\n  }\n});\n```\n\n`findWithReq()` also handles basic security such as making sure the `limit` and `fields` requested on the URL are within the allowed values you specify in `params`.\n\n### Number of results\n\nIf the `limit` parameter isn't passed, then this library will default to returning 50 results. This can be overridden by setting `mongoPaging.config.DEFAULT_LIMIT = \u003cnew default limit\u003e;`. Regardless of the `limit` passed in, a maximum of 300 documents will be returned. This can be overridden by setting `mongoPaging.config.MAX_LIMIT = \u003cnew max limit\u003e;`.\n\n### Alphabetical sorting\n\nThe collation to use for alphabetical sorting, both with `find` and `aggregate`, can be selected by setting `mongoPaging.config.COLLATION`. If this parameter is\nnot set, no collation will be provided for the aggregation/cursor, which means that MongoDB will use whatever collation was set for the collection.\n\n## (!) Important note regarding collation (!)\n\nIf using a global collation setting, or a query with collation argument, ensure that your collections' indexes (that index upon string fields)\nhave been created with the same collation option; if this isn't the case, your queries will be unable to\ntake advantage of any indexes.\n\nSee mongo documentation: https://docs.mongodb.com/manual/reference/collation/#collation-and-index-use\n\nFor instance, given the following index:\n\n```js\ndb.people.createIndex({ city: 1, _id: 1 });\n```\n\nWhen executing the query:\n\n```js\nMongoPaging.find(db.people, {\n  limit: 25,\n  paginatedField: 'city'\n  collation: { locale: 'en' },\n});\n```\n\nThe index won't be used for the query because it doesn't include the collation used. The index should added as such:\n\n```js\ndb.people.createIndex({ city: 1, _id: 1 }, { collation: { locale: 'en' } });\n```\n\n### Indexes for sorting\n\n`mongo-cursor-pagination` uses `_id` as a secondary sorting field when providing a `paginatedField` property. It is recommended that you have an index for optimal performance. Example:\n\n```js\nMongoPaging.find(db.people, {\n  query: {\n    name: 'John'\n  },\n  paginatedField: 'city'\n  limit: 25,\n}).then((results) =\u003e {\n  // handle results.\n});\n```\n\nFor the above query to be optimal, you should have an index like:\n\n```js\ndb.people.createIndex({\n  name: 1,\n  city: 1,\n  _id: 1,\n});\n```\n\n## Running tests\n\nTo run tests, you first must [start a Mongo server on port 27017](https://mongodb.github.io/node-mongodb-native/2.2/quick-start/) and then run `npm test`.\n\n## TODOs\n\n- Convert to typescript and add proper typing\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftawk%2Fmongo-cursor-pagination","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftawk%2Fmongo-cursor-pagination","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftawk%2Fmongo-cursor-pagination/lists"}