{"id":17054580,"url":"https://github.com/grantcarthew/node-rethinkdb-job-queue","last_synced_at":"2025-08-10T08:42:30.905Z","repository":{"id":56254237,"uuid":"58178576","full_name":"grantcarthew/node-rethinkdb-job-queue","owner":"grantcarthew","description":"A persistent job or task queue backed by RethinkDB.","archived":false,"fork":false,"pushed_at":"2020-11-18T15:45:20.000Z","size":957,"stargazers_count":157,"open_issues_count":11,"forks_count":16,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-07-21T04:07:06.220Z","etag":null,"topics":["distributed","job","message","nodejs","npm","persistent","queue","rethinkdb","task"],"latest_commit_sha":null,"homepage":"https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/grantcarthew.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-05-06T03:33:05.000Z","updated_at":"2024-11-30T05:33:27.000Z","dependencies_parsed_at":"2022-08-15T15:31:14.871Z","dependency_job_id":null,"html_url":"https://github.com/grantcarthew/node-rethinkdb-job-queue","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/grantcarthew/node-rethinkdb-job-queue","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grantcarthew%2Fnode-rethinkdb-job-queue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grantcarthew%2Fnode-rethinkdb-job-queue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grantcarthew%2Fnode-rethinkdb-job-queue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grantcarthew%2Fnode-rethinkdb-job-queue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/grantcarthew","download_url":"https://codeload.github.com/grantcarthew/node-rethinkdb-job-queue/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grantcarthew%2Fnode-rethinkdb-job-queue/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269698514,"owners_count":24461195,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-10T02:00:08.965Z","response_time":71,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed","job","message","nodejs","npm","persistent","queue","rethinkdb","task"],"created_at":"2024-10-14T10:15:10.872Z","updated_at":"2025-08-10T08:42:30.864Z","avatar_url":"https://github.com/grantcarthew.png","language":"JavaScript","funding_links":["https://www.patreon.com/grantcarthew"],"categories":["JavaScript"],"sub_categories":[],"readme":"# Introduction\r\n\r\n`rethinkdb-job-queue` is a persistent job or task queue backed by [RethinkDB][rethinkdb-url].\r\nIt has been built as an alternative to the many queues available on [NPM][npm-search-url].\r\n\r\n[![bitHound Overall Score][bithound-overall-image]][bithound-overall-url]\r\n[![bitHound Dependencies][bithound-dep-image]][bithound-dep-url]\r\n[![bitHound Dependencies][bithound-code-image]][bithound-code-url]\r\n[![js-standard-style][js-standard-image]][js-standard-url]\r\n[![NSP Status][nsp-image]][nsp-url]\r\n[![Patreon Donation][patreon-image]][patreon-url]\r\n\r\n[![Thinker][thinker-image]][rjq-github-url]\r\n\r\n[![NPM][nodei-npm-image]][nodei-npm-url]\r\n\r\nPlease __Star__ on GitHub / NPM and __Watch__ for updates.\r\n\r\n## Features\r\n\r\n*   Powered by [RethinkDB][rethinkdb-url]\r\n*   Connect to [multiple databases][queue-connection-url]\r\n*   Create [multiple queues][queue-connection-url]\r\n*   [Distributed worker nodes (PubSub)][queue-pubsub-url]\r\n*   Global [queue events][queue-events-url]\r\n*   Global [job cancellation][job-cancel-url]\r\n*   Global [pause queue][queue-pause-url]\r\n*   Run [concurrent jobs][queue-concurrency-url]\r\n*   Promise based with minimal callbacks\r\n*   [Priority processing of jobs][job-priority-url]\r\n*   [Job progress updates][job-progress-url]\r\n*   [Delayed job start][job-delayed-url]\r\n*   [Find jobs in the queue][find-job-url]\r\n*   [Determine job uniqueness][find-job-by-name-url]\r\n*   [Job timeout][job-timeout-url]\r\n*   [Retrying failed jobs][job-retry-url]\r\n*   [Repeatable jobs][job-repeat-url]\r\n*   [Job reanimation][job-reanimation-url]\r\n*   [Job Editing][job-editing-url]\r\n*   Rich job [history log][job-log-url]\r\n*   TypeScript definitions included\r\n*   Over 2000 [integration tests][testing-url]\r\n\r\n[queue-connection-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Queue-Connection\r\n[queue-events-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Queue-Events\r\n[queue-concurrency-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Queue-Options#queue-concurrency-option\r\n[queue-pubsub-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Queue-PubSub\r\n[job-priority-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Job-Options#job-priority-option\r\n[job-progress-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Job.updateProgress\r\n[job-delayed-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Delayed-Job\r\n[find-job-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Queue.findJob\r\n[find-job-by-name-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Queue.findJobByName\r\n[job-cancel-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Queue.process#failed-job-with-cancel\r\n[queue-pause-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Queue.pause\r\n[job-timeout-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Job-Options#job-timeout-option\r\n[job-retry-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Job-Retry\r\n[job-repeat-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Job.repeat\r\n[job-reanimation-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Queue.reanimateJob\r\n[job-editing-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Job-Editing\r\n[job-log-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Job.log\r\n[testing-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Testing\r\n\r\n## Documentation\r\n\r\n*   [Full Documentation][rjq-wiki-url]\r\n*   [Change Log][rjq-changelog-url]\r\n*   [Code Coverage Report][rjq-coverage-url] from more than 2000 Integration Tests\r\n\r\n## Quick Start\r\n\r\n### Installation\r\n\r\n__Note:__ You will need to install [RethinkDB][rethinkdb-url] before you can use `rethinkdb-job-queue`.\r\n\r\n```sh\r\nnpm install rethinkdb-job-queue --save\r\n```\r\n\r\nSee the [installation document](https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Installation) for supported Node.js versions and workarounds.\r\n\r\n### Simple Example\r\n\r\n```js\r\n\r\nconst Queue = require('rethinkdb-job-queue')\r\nconst qOptions = {\r\n  name: 'Mathematics' // The queue and table name\r\n}\r\nconst cxnOptions = {\r\n  db: 'JobQueue', // The name of the database in RethinkDB\r\n}\r\n\r\nconst q = new Queue(cxnOptions, qOptions)\r\n\r\nconst job = q.createJob({\r\n  numerator: 123,\r\n  denominator: 456\r\n})\r\n\r\nq.process((job, next) =\u003e {\r\n    try {\r\n      let result = job.numerator / job.denominator\r\n      // Do something with your result\r\n      return next(null, result)\r\n    } catch (err) {\r\n      console.error(err)\r\n      return next(err)\r\n    }\r\n})\r\n\r\nreturn q.addJob(job).catch((err) =\u003e {\r\n  console.error(err)\r\n})\r\n\r\n```\r\n\r\n### E-Mail Job Example using [nodemailer][nodemailer-url]\r\n\r\n```js\r\n\r\n// The following is not related to rethinkdb-job-queue.\r\n// This is the nodemailer configuration\r\nconst nodemailer = require('nodemailer')\r\nconst transporter = nodemailer.createTransport({\r\n  service: 'Mailgun',\r\n  auth: {\r\n    user: 'postmaster@superheros.com',\r\n    pass: 'your-api-key-here'\r\n  }\r\n})\r\n\r\n// Setup e-mail data with unicode symbols\r\nvar mailOptions = {\r\n  from: '\"Registration\" \u003csupport@superheros.com\u003e', // Sender address\r\n  subject: 'Registration', // Subject line\r\n  text: 'Click here to complete your registration', // Plaintext body\r\n  html: '\u003cb\u003eClick here to complete your registration\u003c/b\u003e' // HTML body\r\n}\r\n\r\n// rethinkdb-job-queue configuration\r\nconst Queue = require('rethinkdb-job-queue')\r\n\r\n// Queue options have defaults and are not required\r\nconst qOptions = {\r\n  name: 'RegistrationEmail', // The queue and table name\r\n  masterInterval: 310000, // Database review period in milliseconds\r\n  changeFeed: true, // Enables events from the database table\r\n  concurrency: 100,\r\n  removeFinishedJobs: 2592000000, // true, false, or number of milliseconds\r\n}\r\n\r\n// Connection options have defaults and are not required\r\n// You can replace these options with a rethinkdbdash driver object\r\nconst cxnOptions = {\r\n  host: 'localhost',\r\n  port: 28015,\r\n  db: 'JobQueue', // The name of the database in RethinkDB\r\n}\r\n\r\n// This is the main queue instantiation call\r\nconst q = new Queue(cxnOptions, qOptions)\r\n\r\n// Customizing the default job options for new jobs\r\nq.jobOptions = {\r\n  priority: 'normal',\r\n  timeout: 300000,\r\n  retryMax: 3, // Four attempts, first then three retries\r\n  retryDelay: 600000 // Time in milliseconds to delay retries\r\n}\r\n\r\nconst job = q.createJob()\r\n// The createJob method will only create the job locally.\r\n// It will need to be added to the queue.\r\n// You can decorate the job with any data to be saved for processing\r\njob.recipient = 'batman@batcave.com'\r\n\r\nq.process((job, next) =\u003e {\r\n  // Send email using job.recipient as the destination address\r\n  mailOptions.to = job.recipient\r\n  return transporter.sendMail(mailOptions).then((info) =\u003e {\r\n    console.dir(info)\r\n    return next(null, info)\r\n  }).catch((err) =\u003e {\r\n    // This catch is for nodemailer sendMail errors.\r\n    return next(err)\r\n  })\r\n})\r\n\r\nreturn q.addJob(job).then((savedJobs) =\u003e {\r\n  // savedJobs is an array of the jobs added with updated properties\r\n}).catch((err) =\u003e {\r\n  console.error(err)\r\n})\r\n\r\n```\r\n\r\n## About the Owner\r\n\r\nI, Grant Carthew, am a technologist, trainer, and Dad from Queensland, Australia. I work on code in a number of personal projects and when the need arises I build my own packages.\r\n\r\nThis project exists because there were no functional job queues built on the RethinkDB database. I wanted an alternative to the other job queues on NPM.\r\n\r\nEverything I do in open source is done in my own time and as a contribution to the open source community.\r\n\r\nIf you are using my projects and would like to thank me or support me, please click the Patreon link below.\r\n\r\n[![Patreon Donation][patreon-image]][patreon-url]\r\n\r\nSee my [other projects on NPM](https://www.npmjs.com/~grantcarthew).\r\n\r\n## Contributing\r\n\r\n1.  Fork it!\r\n2.  Create your feature branch: `git checkout -b my-new-feature`\r\n3.  Commit your changes: `git commit -am 'Add some feature'`\r\n4.  Push to the branch: `git push origin my-new-feature`\r\n5.  Submit a pull request :D\r\n\r\nWant to help but not sure how? Checkout the [ideas](https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Ideas) page.\r\n\r\nPlease see the [debugging](https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Debugging) and [testing](https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Testing) documents for more detail.\r\n\r\n## Credits\r\n\r\nThanks to the following marvelous packages and people for their hard work:\r\n\r\n-   The [RethinkDB][rethinkdb-url] team for the great database.\r\n-   The RethinkDB driver [rethinkdbdash][rethinkdbdash-url] by [Michel][neumino-url]\r\n-   The Promise library [Bluebird][bluebird-url] by [Petka Antonov][petka-url].\r\n-   The [uuid][uuid-url] package.\r\n\r\nThis list could go on...\r\n\r\n## License\r\n\r\nMIT\r\n\r\n[npm-search-url]: https://npms.io/search?q=job+queue\r\n[rethinkdb-url]: http://www.rethinkdb.com/\r\n[rethinkdbdash-url]: https://github.com/neumino/rethinkdbdash\r\n[neumino-url]: https://github.com/neumino\r\n[rjq-github-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue\r\n[rjq-wiki-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki\r\n[rjq-changelog-url]: https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki/Change-Log\r\n[rjq-coverage-url]: https://rawgit.com/grantcarthew/site-rjq-coverage/master/lcov-report/src/index.html\r\n[thinker-image]: https://cdn.rawgit.com/grantcarthew/node-rethinkdb-job-queue/master/thinkerjoblist.png\r\n[nodemailer-url]: https://www.npmjs.com/package/nodemailer\r\n[bluebird-url]: https://github.com/petkaantonov/bluebird\r\n[petka-url]: https://github.com/petkaantonov\r\n[uuid-url]: https://www.npmjs.com/package/uuid\r\n[bithound-overall-image]: https://www.bithound.io/github/grantcarthew/node-rethinkdb-job-queue/badges/score.svg\r\n[bithound-overall-url]: https://www.bithound.io/github/grantcarthew/node-rethinkdb-job-queue\r\n[bithound-dep-image]: https://www.bithound.io/github/grantcarthew/node-rethinkdb-job-queue/badges/dependencies.svg\r\n[bithound-dep-url]: https://www.bithound.io/github/grantcarthew/node-rethinkdb-job-queue/master/dependencies/npm\r\n[bithound-code-image]: https://www.bithound.io/github/grantcarthew/node-rethinkdb-job-queue/badges/code.svg\r\n[bithound-code-url]: https://www.bithound.io/github/grantcarthew/node-rethinkdb-job-queue\r\n[js-standard-image]: https://img.shields.io/badge/code%20style-standard-brightgreen.svg\r\n[js-standard-url]: http://standardjs.com/\r\n[nsp-image]: https://nodesecurity.io/orgs/openjs/projects/0244545a-bc28-4903-b639-ec5468d7146d/badge\r\n[nsp-url]: https://nodesecurity.io/orgs/openjs/projects/0244545a-bc28-4903-b639-ec5468d7146d\r\n[patreon-image]: https://img.shields.io/badge/patreon-donate-yellow.svg\r\n[patreon-url]: https://www.patreon.com/grantcarthew\r\n[nodei-npm-image]: https://nodei.co/npm/rethinkdb-job-queue.png?downloads=true\u0026downloadRank=true\u0026stars=true\r\n[nodei-npm-url]: https://nodei.co/npm/rethinkdb-job-queue/\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrantcarthew%2Fnode-rethinkdb-job-queue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgrantcarthew%2Fnode-rethinkdb-job-queue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrantcarthew%2Fnode-rethinkdb-job-queue/lists"}