{"id":20485283,"url":"https://github.com/runnable/ponos","last_synced_at":"2025-04-13T14:52:58.926Z","repository":{"id":36827132,"uuid":"41134020","full_name":"Runnable/ponos","owner":"Runnable","description":"An opinionated queue based worker server for node.","archived":false,"fork":false,"pushed_at":"2018-10-27T15:47:22.000Z","size":890,"stargazers_count":86,"open_issues_count":23,"forks_count":8,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-03-27T05:51:11.228Z","etag":null,"topics":["rabbitmq","worker-server","workers"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Runnable.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-08-21T04:08:01.000Z","updated_at":"2022-04-11T20:05:50.000Z","dependencies_parsed_at":"2022-07-20T11:02:02.558Z","dependency_job_id":null,"html_url":"https://github.com/Runnable/ponos","commit_stats":null,"previous_names":[],"tags_count":50,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Runnable%2Fponos","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Runnable%2Fponos/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Runnable%2Fponos/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Runnable%2Fponos/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Runnable","download_url":"https://codeload.github.com/Runnable/ponos/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248732509,"owners_count":21152851,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["rabbitmq","worker-server","workers"],"created_at":"2024-11-15T16:29:11.011Z","updated_at":"2025-04-13T14:52:58.907Z","avatar_url":"https://github.com/Runnable.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Ponos\n\n[![travis]](https://travis-ci.org/Runnable/ponos)\n[![coveralls]](https://coveralls.io/github/Runnable/ponos?branch=master)\n[![dependencies]](https://david-dm.org/Runnable/ponos)\n[![devdependencies]](https://david-dm.org/Runnable/ponos#info=devDependencies)\n[![codeclimate]](https://codeclimate.com/github/Runnable/ponos)\n\nDocumentation is available at [runnable.github.io/ponos][documentation]\n\nA migration guide for v3.0.0 [is available](docs/Guides-Migration-v3.0.0.md)!\n\nAn opinionated queue based worker server for node.\n\nFor ease of use we provide options to set the host, port, username, and password to the RabbitMQ server. If not present in options, the server will attempt to use the following environment variables and final defaults:\n\noptions                             | environment                 | default\n------------------------------------|-----------------------------|--------------\n`opts.rabbitmq.hostname`            | `RABBITMQ_HOSTNAME`         | `'localhost'`\n`opts.rabbitmq.port`                | `RABBITMQ_PORT`             | `'5672'`\n`opts.rabbitmq.username`            | `RABBITMQ_USERNAME`         | _none_\n`opts.rabbitmq.password`            | `RABBITMQ_PASSWORD`         | _none_\n`opts.redisRateLimiting.host`       | `REDIS_HOST`                | `'localhost'`\n`opts.redisRateLimiting.port`       | `REDIS_PORT`                | `'6379'`\n`opts.redisRateLimiting.durationMs` | `RATE_LIMIT_DURATION`       | `1000`\n`opts.log`                          | _N/A_                       | Basic [bunyan](https://github.com/trentm/node-bunyan) instance with `stdout` stream (for logging)\n`opts.errorCat`                     | _N/A_                       | Basic [error-cat](https://github.com/runnable/error-cat) instance (for rollbar error reporting)\n\nOther options for Ponos are as follows:\n\nenvironment variable     | default | description\n-------------------------|---------|------------\n`WORKER_MAX_RETRY_DELAY` | `0`     | The maximum time, in milliseconds, that the worker will wait before retrying a task. The timeout will exponentially increase from `MIN_RETRY_DELAY` to `MAX_RETRY_DELAY` if the latter is set higher than the former. If this value is not set, the worker will not exponentially back off.\n`WORKER_MIN_RETRY_DELAY` | `1`     | Time, in milliseconds, the worker will wait at minimum will wait before retrying a task.\n`WORKER_TIMEOUT`         | `0`     | Timeout, in milliseconds, at which the worker task will be retried.\n`WORKER_MAX_NUM_RETRIES` | `0`     | The maximum number of times a job will retry due to failures. 0 means infinity\n\n## Usage\n\nFrom a high level, Ponos is used to create a worker server that responds to jobs provided from RabbitMQ. The user defines handlers for each queue's jobs that are invoked by Ponos.\n\nPonos has built in support for retrying and catching specific errors, which are described below.\n\n## Workers\n\nWorkers need to be defined as a function that takes a Object `job` an returns a promise. For example:\n\n```javascript\nfunction myWorker (job) {\n  return Promise.resolve()\n    .then(() =\u003e {\n      return doSomeWork(job)\n    })\n}\n```\n\nThis worker takes the `job`, does work with it, and returns the result. Since (in theory) this does not throw any errors, the worker will see this resolution and acknowledge the job.\n\n## Tasks vs. Events\n\nPonos provides (currently) two paradigms for doing work. First is subscribing directly to queues in RabbitMQ using the `tasks` parameter in the constructor. The other is the ability to subscribe to a fanout exchange using the `events` parameter, which can provide for a much more useful utilization of RabbitMQ's structure.\n\n```javascript\nconst ponos = require('ponos')\nconst server = new ponos.Server({\n  tasks: {\n    'a-queue': (job) =\u003e { return Promise.resolve(job) }\n  },\n  events: {\n    'an-exchange': (job) =\u003e { return Promise.resolve(job) }\n  }\n})\n```\n\n### Worker Errors\n\nPonos's worker is designed to retry any error that is not specifically a fatal error. Ponos has been designed to work well with our error library [`error-cat`](https://github.com/Runnable/error-cat).\n\nA fatal error is defined with the `WorkerStopError` class from `error-cat`. If a worker rejects with a `WorkerStopError`, the worker will automatically assume the job can _never_ be completed and will acknowledge the job.\n\nAs an example, a `WorkerStopError` can be used to fail a task given an invalid job:\n\n```javascript\nconst WorkerStopError = require('error-cat/errors/worker-stop-error')\nfunction myWorker (job) {\n  return Promise.resolve()\n    .then(() =\u003e {\n      if (!job.host) {\n        throw new WorkerStopError('host is required', {}, 'my.queue', job)\n      }\n    })\n    .then(() =\u003e {\n      return doSomethingWithHost(job)\n    })\n}\n```\n\nThis worker will reject the promise with a `WorkerStopError`. Ponos will log the error itself, acknowledge the job to remove it from the queue, and continue with other jobs. You may catch and re-throw the error if you wish to do additional logging or reporting.\n\nFinally, as was mentioned before, Ponos will retry any other errors. `error-cat` provides a `WorkerError` class you may use, or you may throw normal `Error`s. If you do, the worker will catch these and retry according to the server's configuration (retry delay, back-off, max delay, etc.).\n\n```javascript\nconst WorkerError = require('error-cat/errors/worker-error')\nconst WorkerStopError = require('error-cat/errors/worker-stop-error')\nfunction myWorker (job) {\n  return Promise.resolve()\n    .then(() =\u003e {\n      return externalService.something(job)\n    })\n    // Note: w/o this catch, the error would simply propagate to the worker and\n    // be handled.\n    .catch((err) =\u003e {\n      logErrorToService(err)\n      // If the error is 'recoverable' (e.g., network fluke, temporary outage),\n      // we want to be able to retry.\n      if (err.isRecoverable) {\n        throw new Error('hit a momentary issue. try again.')\n      } else {\n        // maybe we know we can't recover from this\n        throw new WorkerStopError(\n          'cannot recover. acknowledge and remove job',\n          {},\n          'this.queue',\n          job\n        )\n      }\n    })\n}\n```\n\n## Worker Options\n\nCurrently workers can be defined with a `msTimeout` option. This value defaults to `process.env.WORKER_TIMEOUT || 0`. One can set a specific millisecond timeout for a worker like so:\n\n```js\nserver.setTask('my-queue', workerFunction, { msTimeout: 1234 })\n```\n\nOr one can set this option via `setAllTasks`:\n\n```js\nserver.setAllTasks({\n  // This will use the default timeout, maxNumRetries, ...\n  'queue-1': queueOneTaskFn,\n  // This will use the specified timeout, maxNumRetries, ...\n  'queue-2': {\n    // worker function to run\n    task: queueTwoTaskFn,\n\n    // schema to validate job against\n    jobSchema: Joi.object({ tid: Joi.string() }),\n\n    // time before job will throw timeout error\n    msTimeout: 1337,\n\n    // number of times before job will stop retrying on failure\n    maxNumRetries: 7,\n\n    // function to run when we hit max retries\n    finalRetryFn: () =\u003e { return Promise.try(...)},\n\n    // number of jobs that we can perform in given duration\n    maxOperations: 5,\n\n    // duration under which rate limit is accounted for\n    durationMs: 60000\n  }\n})\n```\n\nThese options are also available for `setEvent` and `setAllEvents`.\n\n## Worker Namespaces\n\nEach worker is wrapped in a [continuation-local-storage](https://github.com/othiym23/node-continuation-local-storage) namespace called `ponos`.\n\nPonos adds a `tid` to the `ponos` namespace. This `tid` is unique per job. To access this `tid`:\n\n```js\nconst getNamespace = require('continuation-local-storage').getNamespace\n\nmodule.export.worker = Promise.try(() =\u003e {\n  const tid = getNamespace('ponos').get('tid')\n  console.log(`hello world: tid: ${tid}`)\n})\n```\n\n**NOTES:**\n* `Promise.resolve().then(() =\u003e {...})` breaks out of Ponos namespace and `tid` will not be available\n* `getNamespace` must be called in the worker itself\n\n## Full Example\n\n```javascript\nconst ponos = require('ponos')\n\nconst tasks = {\n  'queue-1': (job) =\u003e { return Promise.resolve(job) },\n  'queue-2': (job) =\u003e { return Promise.resolve(job) }\n}\n\nconst events = {\n  'exchange-1': (job) =\u003e { return Promise.resolve(job) }\n}\n\n// Create the server\nvar server = new ponos.Server({\n  events: events,\n  tasks: tasks\n})\n\n// If tasks were not provided in the constructor, set tasks for workers handling\n// jobs on each queue\nserver.setAllTasks(tasks)\n// Similarly, you can set events.\nserver.setAllEvents(events)\n\n// Start the server!\nserver.start()\n  .then(() =\u003e { console.log('Server started!') })\n  .catch((err) =\u003e { console.error('Server failed', err) })\n```\n\n## License\n\nMIT\n\n[travis]: https://img.shields.io/travis/Runnable/ponos/master.svg?style=flat-square \"Build Status\"\n[coveralls]: https://img.shields.io/coveralls/Runnable/ponos/master.svg?style=flat-square \"Coverage Status\"\n[dependencies]: https://img.shields.io/david/Runnable/ponos.svg?style=flat-square \"Dependency Status\"\n[devdependencies]: https://img.shields.io/david/dev/Runnable/ponos.svg?style=flat-square \"Dev Dependency Status\"\n[documentation]: https://runnable.github.io/ponos \"Ponos Documentation\"\n[codeclimate]: https://img.shields.io/codeclimate/github/Runnable/ponos.svg?style=flat-square \"Code Climate\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frunnable%2Fponos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frunnable%2Fponos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frunnable%2Fponos/lists"}