{"id":19020257,"url":"https://github.com/trailsjs/trailpack-tasker","last_synced_at":"2025-04-23T05:27:30.064Z","repository":{"id":57378880,"uuid":"57185873","full_name":"trailsjs/trailpack-tasker","owner":"trailsjs","description":":package: Easily set up background workers with a Message Broker (e.g. RabbitMQ, Kafka)","archived":false,"fork":false,"pushed_at":"2017-03-24T14:50:26.000Z","size":33,"stargazers_count":3,"open_issues_count":4,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-17T20:40:03.629Z","etag":null,"topics":["kafka","rabbitmq","trailpack-tasker","trails"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/trailsjs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-04-27T05:27:34.000Z","updated_at":"2017-11-08T16:36:31.000Z","dependencies_parsed_at":"2022-09-05T08:10:47.581Z","dependency_job_id":null,"html_url":"https://github.com/trailsjs/trailpack-tasker","commit_stats":null,"previous_names":["langateam/trailpack-tasker"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trailsjs%2Ftrailpack-tasker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trailsjs%2Ftrailpack-tasker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trailsjs%2Ftrailpack-tasker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trailsjs%2Ftrailpack-tasker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/trailsjs","download_url":"https://codeload.github.com/trailsjs/trailpack-tasker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250376284,"owners_count":21420306,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kafka","rabbitmq","trailpack-tasker","trails"],"created_at":"2024-11-08T20:16:21.473Z","updated_at":"2025-04-23T05:27:30.022Z","avatar_url":"https://github.com/trailsjs.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# trailpack-tasker\n\n[![NPM version][npm-image]][npm-url]\n[![Build status][ci-image]][ci-url]\n[![Dependency Status][daviddm-image]][daviddm-url]\n[![Code Climate][codeclimate-image]][codeclimate-url]\n\nEasily set up background workers with [RabbitMQ](https://www.rabbitmq.com/) and [Trails](http://trailsjs.io).\nThis project is built on top of the [rabbot](https://github.com/arobson/rabbot) RabbitMQ client.\n\n## Install\n\n```sh\n$ npm install --save trailpack-tasker\n```\n\n## Configure\n\n### Add Trailpack\n\n```js\n// config/main.js\nmodule.exports = {\n  packs: [\n    // ... other trailpacks\n    require('trailpack-tasker')\n  ]\n}\n```\n\n### Configure Tasker Settings\n\n```js\n// config/tasker.js\nmodule.exports = {\n\n  /**\n   * Define worker profiles. Each worker of a given type listens for the\n   * \"tasks\" defined in its profile below. The task names represent a Task\n   * defined in api.services.tasks. Note that 'memoryBound' and 'cpuBound' are\n   * arbitrary names.\n   */\n  profiles: {\n    memoryBound: {\n      tasks: [ 'hiMemoryTask1' ]\n    },\n    cpuBound: {\n      tasks: [ 'VideoEncoder', 'hiCpuTask2' ]\n    }\n  },\n\n  /**\n   * Set RabbitMQ connection info\n   */\n  connection: {\n    exchange: process.env.TASKER_EXCHANGE, // optional, defaults to `tasker-work-x`\n    workQueueName: process.env.TASKER_WORK_QUEUE, // optional, defaults to `tasker-work-q`\n    interruptQueueName: process.env.TASKER_INTERRUPT_QUEUE, // optional, defaults to `tasker-interrupt-q`\n\n    /**\n     * The RabbitMQ connection information.\n     * See: https://www.rabbitmq.com/uri-spec.html\n     */\n     host: process.env.TASKER_RMQ_HOST,\n     user: process.env.TASKER_RMQ_USER,\n     pass: process.env.TASKER_RMQ_PASS,\n     port: process.env.TASKER_RMQ_PORT,\n     vhost: process.env.TASKER_RMQ_VHOST\n\n     /**\n     * Connection information could also be passed via uri\n     */\n     uri: process.env.TASKER_RMQ_URI\n\n     /**\n      * Additional, optional connection options (default values shown)\n      */\n      heartbeat: 30,\n      timeout:, // this is the connection timeout (in milliseconds, per connection attempt), and there is no default\n      failAFter: 60, // limits how long rabbot will attempt to connect (in seconds, across all connection attempts). Defaults to 60\n      retryLimit: 3, // limits number of consecutive failed attempts\n\n  },\n\n  /**\n   * Set worker to subscribe to tasks in the matching profile (tasker.profiles).\n   * If process.env.WORKER does not match a profile, the application will not subscribe to any tasks\n   */\n  worker: process.env.WORKER\n}\n```\n\n### Configure `worker` Environment\n\n```js\n// config/env/worker.js\nmodule.exports = {\n  main: {\n\n    /**\n     * Only load the packs needed by the workers\n     */\n    packs: [\n      require('trailpack-core'),\n      require('trailpack-tasker')\n    ]\n  }\n}\n```\n\nIf the worker profiles each require more granular environment configurations,\ncreate `worker-cpuBound`, `worker-memoryBound`, etc. environments.\n\n### Include tasks in the app object\nCreate a directory `api/tasks`.  Any task definitions will be created as classes in this directory.\nCreate  `api/tasks/index.js` to export all of the tasks.\nInclude this directory in `api/index.js`.  Here is an example:\n```js\n// api/index.js\n\nexports.controllers = require('./controllers')\nexports.models = require('./models')\nexports.policies = require('./policies')\nexports.services = require('./services')\nexports.tasks = require('./tasks')\n```\n\n## Usage\n\nDefine tasks in `api.tasks`.  Tasks are run by a worker processes.\n\n```js\n// api/tasks/VideoEncoder.js\n\nconst Task = require('trailpack-tasker').Task\nmodule.exports = class VideoEncoder extends Task {\n\n  /**\n   * \"message\" is the message from RabbitMQ, and contains all the information\n   * the worker needs to do its job. By default, sets this.message and this.app.\n   *\n   * @param message.body.videoFormat\n   * @param message.body.videoBuffer\n   */\n  constructor (app, message) {\n    super(app, message)\n  }\n\n  /**\n   * Do work here. When the work is finished (the Promise is resolved), send\n   * \"ack\" to the worker queue. You must override this method.\n   *\n   * @return Promise\n   */\n  run () {\n    return doWork(this.message)\n  }\n\n  /**\n   * This is a listener which is invoked when the worker is interrupted (specifically,\n   * an interrupt is a particular type of message that instructs this worker to\n   * stop).\n   */\n  interrupt () {\n    this.log.warn('only encoded', this.currentIndex, 'out of', this.totalItems, 'frames')\n  }\n\n  /**\n   * Perform any necessary cleanup, close connections, etc. This method will be\n   * invoked regardless of whether the worker completed successfully or not.\n   * @return Promise\n   */\n  finalize () {\n    return doCleanup(this.message)\n  }\n}\n```\n\n\nTo start a task, publish a message via the `app.tasker` interface: \n```\nconst taskId = app.tasker.publish('VideoEncoder', { vidoeUrl: 'http://...' }\n```\n\nTo interrupt a task in progress, use the `taskId` that is returned from app.tasker.publish(..):\n```\napp.tasker.cancel('VideoEncoder', taskId)\n```\n\n\n\n## Deployment\n\nAn example [Procfile](https://devcenter.heroku.com/articles/procfile) may look like:\n\n```\nweb: npm start\nmemoryBound: NODE_ENV=worker WORKER=memoryBound npm start\ncpuBound: NODE_ENV=worker WORKER=cpuBound npm start\n```\n\n\n## License\nMIT\n\n## Maintained By\n[\u003cimg src='http://i.imgur.com/Y03Jgmf.png' height='64px'\u003e](https://langa.io)\n\n[npm-image]: https://img.shields.io/npm/v/trailpack-tasker.svg?style=flat-square\n[npm-url]: https://npmjs.org/package/trailpack-tasker\n[ci-image]: https://img.shields.io/travis/langateam/trailpack-tasker/master.svg?style=flat-square\n[ci-url]: https://travis-ci.org/langateam/trailpack-tasker\n[daviddm-image]: http://img.shields.io/david/langateam/trailpack-tasker.svg?style=flat-square\n[daviddm-url]: https://david-dm.org/langateam/trailpack-tasker\n[codeclimate-image]: https://img.shields.io/codeclimate/github/langateam/trailpack-tasker.svg?style=flat-square\n[codeclimate-url]: https://codeclimate.com/github/langateam/trailpack-tasker\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrailsjs%2Ftrailpack-tasker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftrailsjs%2Ftrailpack-tasker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrailsjs%2Ftrailpack-tasker/lists"}