{"id":49251595,"url":"https://github.com/not-empty/taurus-queue-nodejs","last_synced_at":"2026-04-25T00:41:56.511Z","repository":{"id":42570673,"uuid":"281766022","full_name":"not-empty/taurus-queue-nodejs","owner":"not-empty","description":"Simple Node.js App to manage queues on redis based on Bull","archived":false,"fork":false,"pushed_at":"2024-07-23T14:16:40.000Z","size":116,"stargazers_count":16,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-12T05:36:55.459Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/not-empty.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-22T19:30:31.000Z","updated_at":"2024-07-23T14:14:45.000Z","dependencies_parsed_at":"2024-07-23T16:19:57.550Z","dependency_job_id":"11f37d78-76c4-44b2-a99c-038106f9dafb","html_url":"https://github.com/not-empty/taurus-queue-nodejs","commit_stats":null,"previous_names":["not-empty/taurus-queue-nodejs"],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/not-empty/taurus-queue-nodejs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/not-empty%2Ftaurus-queue-nodejs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/not-empty%2Ftaurus-queue-nodejs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/not-empty%2Ftaurus-queue-nodejs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/not-empty%2Ftaurus-queue-nodejs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/not-empty","download_url":"https://codeload.github.com/not-empty/taurus-queue-nodejs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/not-empty%2Ftaurus-queue-nodejs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32246405,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T13:21:15.438Z","status":"ssl_error","status_checked_at":"2026-04-24T13:21:15.005Z","response_time":64,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-25T00:41:55.934Z","updated_at":"2026-04-25T00:41:56.502Z","avatar_url":"https://github.com/not-empty.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Taurus Queue\n\n[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)\n\n## Overview\n\nTaurus Queue offers a comprehensive queue ecosystem, simplifying the creation, execution, management, and monitoring of scalable and highly available queues. Leveraging the robust foundation of the [Bull Project](https://github.com/OptimalBits/bull), Taurus Queue eliminates the intricacies of queue coding, allowing you to concentrate solely on your specific actions and rules, thus optimizing your time. It features a dedicated interface for efficient queue management and monitoring.\n\nGroups and queues list\n![Taurus Manager Queue list](https://github.com/not-empty/taurus-manager-vue/blob/master/screenshot1.png)\n\nQueue details\n![Taurus Manager Queue list](https://github.com/not-empty/taurus-manager-vue/blob/master/screenshot2.png)\n\nMonitor Screen (Monitoring only unhealth queues with autorefresh)\n![Taurus Manager Queue list](https://github.com/not-empty/taurus-manager-vue/blob/master/screenshot3.png)\n\n## How the Queue Ecosystem Operates\n\n### Setup and Job Publishing\n1. Starting in this repository to create and run your first queue.\n2. Publish jobs to your queues using our publishers, compatible with multiple programming languages:\n   - [PHP Taurus Queue Publisher](https://github.com/not-empty/taurus-publisher-php-lib)\n   - [Go Taurus Queue Publisher](https://github.com/not-empty/taurus-publisher-golang)\n   - [Python Taurus Queue Publisher](https://github.com/not-empty/taurus-publisher-python)\n   - [Node.JS, native support with examples included in this repository](https://github.com/not-empty/taurus-queue-nodejs/blob/master/sample/producer.js)\n   - [Use Taurus Manager interface to create new job to an existent queue](https://github.com/not-empty/taurus-manager-vue)\n\n\n### Queue Management with Taurus Manager\n3. Utilize [Taurus Manager](https://github.com/not-empty/taurus-manager-vue) for:\n   - Pausing/unpausing, adding/removing jobs.\n   - Deleting, retrying, debuggin, viewing error logs and much more.\n   - Managing user permissions.\n   - Overseeing your queues.\n\n### Real-Time Monitoring\n4. Implement [Taurus Monitoring](https://github.com/not-empty/taurus-monitoring) for real-time graphical insights of your entire ecosystem, integrating with [Grafana](https://grafana.com) and [Prometheus](https://prometheus.io):\n   - Queue Length\n   - Job Duration\n   - Queue States\n   - Failures by Queue\n   - Total Jobs Completed (All-Time/Periodic)\n   - Sum of Completed Jobs (All-Time/Periodic)\n\n\n### Installation\n\n[Release 1.0.0](https://github.com/not-empty/taurus-queue-nodejs/releases/tag/1.0.0) Requires [NodeJs](https://nodejs.org) 20.x\n\nWhen running Docker your queue will automatically be running and you just need to include your business rules.\n:)\n\n```sh\ndocker-compose up\n```\n### Setting Your Own Queue\n\nYou can use our `default-business.js` located in `business` folder and include your rules / actions inside the try block\n\n```js\n...\ntry {\n...\n      // your actions and rules here\n...\n} catch (error) {\n...\n```\n\nOr you can create your own business (maybe multiple ones in same project, and even push from one queue to another one).\nFor do that:\n\n1 - Copy our `default-business.js` located in `business`, giving your name, in this sample we'll use `myown-business.js`;\n\n2 - Change the name of the class on the top and the bottom of the file\nFrom this:\n```js\n...\nconst BaseBusiness = require('./base-business');\n\n/**\n * Example bussines job processor\n */\nclass MyOwnBusiness extends BaseBusiness { // changed from DefaultBusiness\n...\n\nmodule.exports = MyOwnBusiness; // changed from DefaultBusiness\n```\n\n3 - Include your rules / actions inside the try block\n```js\n...\ntry {\n...\n      // your actions and rules here\n...\n} catch (error) {\n...\n```\n\n4 - Last but not least declare your business in the `constructor.js` file inside the `config` folder\n```js\nconst DefaultBusiness = require('../business/default-business');\nconst MyOwnBusiness = require('../business/myown-business'); // Added\n\nmodule.exports = {\n  'default': DefaultBusiness,\n  'myown': MyOwnBusiness, // Added\n};\n```\n### Running Your Queue Worker\n\nRequires [Redis](https://redis.io/). \n\nYou can start with [Docker](https://docs.docker.com/get-docker/) using compose tool.\n\nIf you want to run the `default` queue just run the docker\n\n```sh\ndocker-compose up\n```\n\nYou can now run your queue worker. (if you want to, just change the ./ops/docker/dev/run.sh to your new business, changing the \"default\" word for your queue name, in this case \"myown\") like this:\n\n```bash\n#!/bin/sh\n\nnpm i\nnpm run dev myown 1\n```\n\nYou can enter in container e run a queue worker by yourself passing your queue name and the debug mode (1 or 0)\n\n```sh\ndocker exec -it taurus-queue bash\n```\n\nRunning a queue worker in debug mode (with node flag --watch to update service when code changes)\n```sh\nnpm run dev default 1\n```\n\nRunning a queue worker in debug mode\n```sh\nnpm start default 1\n```\n\nWith the debug mode on, your queue will generate outputs from the `log.debug` command, make your development and debug easer.\nThe `log.show` command always have output, so use carefully\n\nYou can also run a cluster with multiples queue workers, in this case we running 5 workers.\n* if you choose to do that, take care of your resources like Memory and CPU, it can be very, very heavy depends on the number of works and the operations of your business.\n\nRunning 5 queue workers in debug mode (with node flag --watch to update service when code changes)\n```sh\nnpm dev-cluster default 5 1\n```\n\nRunning 5 queue workers in debug mode\n```sh\nnpm start-cluster default 5\n```\n\n### Pushing Jobs Itens On Your Queue\n\nNow that you have your workers running, it's time to push itens to your queue.\nYou can do that by running the `producer.js` or the `multi-producer.js` files in the example `folder`.\nAs the name sujests, the `producer.js` send one job to the queue and the `multi-producer.js` send multiples.\n\nIn `producer.js` you can pass as parameter the name of the queue, if you not inform the `default` queue will be used.\n\nYou can also pass a JSON with your data. IF you not inform the JSON a default test data will be used.\n\nIf you want to run `producer.js` in the docker:\n\n```sh\ndocker exec taurus-queue node sample/producer.js default '{\"data\":\"mydata\"}'\n```\n\nIf you inside the container or want to run locally:\n```sh\nnode sample/producer.js default '{\"data\":\"mydata\"}'\n```\n\nIn `multi-producer.js` you pass as parameter the name of the queue and the number of jobs.\n\nIf you not inform the name, the `default`queue will be used.\n\nIf you not inform the number of jobs it will use 2.\n\nYou can also pass a JSON with your data. IF you not inform the JSON a default test data will be used.\n\nIf you want to put 60 jobs on the default queue using docker:\n\n```sh\ndocker exec taurus-queue node sample/multi-producer.js default 60 '{\"data\":\"mydata\"}'\n```\n\nIf you inside the container or want to run locally:\n```sh\nnode sample/multi-producer.js default 60 '{\"data\":\"mydata\"}'\n```\n### Discover when all parallel async jobs are done\n\nIf you use parallel processing with multiple workers, finding out when all jobs have completed successfully can be a complicated task due to asynchrony.\n\nTo deal with this, Taurus has a functionality that uses a Redis + Lua solution to ensure that the last job in a group was executed, to use it you will need to set a unique specific key with the amount of total queues you wanto to excecute, before sending to the queues, and decrement this key to each queue that runs successfully.\n\nThe last queue will know it is last and will allow you to perform finishing actions.\n\nYou need:\n\n1) Fill in the Redis data responsible for control in the .env file (we recommend not being the same person who manages the queues)\n\n```\nAUX_REDIS_HOST=taurus-redis\nAUX_REDIS_PORT=6379\n```\n\n2) When inserting into each queue you can use the CheckCompletion class to increment each job: (If you are outside the Taurus ecosystem you can just create a key in redis with [SET command](https://redis.io/commands/set/) on your favorite language but the value MUST BE INTEGER greater than 0.)\n\n```js\nconst CheckCompletion = require('../core/check-completion.js');\n...\nconst numberOfJobs = Number (100);\nconst checkCompletion = new CheckCompletion();\nawait checkCompletion.setInitialJobCounter(uniqueKeyToRepresenTheGroup, numberOfJobs);\n```\n\n3) When each queue has finished executing, just call the decrement command and check if it returned zero, if it returned zero, this queue is the last one of this group and you can execute any command you find necessary\n\n```js\nconst CheckCompletion = require('../core/check-completion.js');\n...\nconst checkCompletion = new CheckCompletion();\nconst taksRemaining = await checkCompletion.decrement(sameUniqueKeyToRepresenTheGroup);\nif (taksRemaining === 0) {\n   console.log('All jobs are finished');\n}\n```\n\n### Development\n\nWant to contribute? Great!\n\nThe project using a simple code.\nMake a change in your file and be careful with your updates!\n**Any new code will only be accepted with all viladations.**\n\nTo ensure that the entire project is fine:\n\nRun all validations\n\n```sh\n$ npm run check\n```\n\n**Not Empty Foundation - Free codes, full minds**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnot-empty%2Ftaurus-queue-nodejs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnot-empty%2Ftaurus-queue-nodejs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnot-empty%2Ftaurus-queue-nodejs/lists"}