{"id":15364435,"url":"https://github.com/xadillax/scarlet-task","last_synced_at":"2025-04-14T03:33:59.456Z","repository":{"id":16743805,"uuid":"19501295","full_name":"XadillaX/scarlet-task","owner":"XadillaX","description":"Scarlet Task is an advanced task queue module for Node.js, featuring the ability to configure multiple child queues for a single task queue.","archived":false,"fork":false,"pushed_at":"2024-09-23T07:28:25.000Z","size":116,"stargazers_count":20,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-27T17:35:48.417Z","etag":null,"topics":["flandre-scarlet","task-queue"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/XadillaX.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-05-06T16:11:28.000Z","updated_at":"2024-09-23T07:28:28.000Z","dependencies_parsed_at":"2024-06-21T19:06:59.108Z","dependency_job_id":"fc1b0d7f-cad6-4993-93e3-3c641a23eed1","html_url":"https://github.com/XadillaX/scarlet-task","commit_stats":{"total_commits":27,"total_committers":3,"mean_commits":9.0,"dds":"0.40740740740740744","last_synced_commit":"58bfe3a4627dd721974568c800ea4cecfa51c1d9"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XadillaX%2Fscarlet-task","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XadillaX%2Fscarlet-task/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XadillaX%2Fscarlet-task/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XadillaX%2Fscarlet-task/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/XadillaX","download_url":"https://codeload.github.com/XadillaX/scarlet-task/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248816252,"owners_count":21166053,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["flandre-scarlet","task-queue"],"created_at":"2024-10-01T13:11:44.562Z","updated_at":"2025-04-14T03:33:59.439Z","avatar_url":"https://github.com/XadillaX.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"scarlet.gif\" alt=\"Scarlet Task\"\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003eScarlet Task\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://coveralls.io/github/XadillaX/scarlet-task?branch=master\"\u003e\n    \u003cimg src=\"https://coveralls.io/repos/github/XadillaX/scarlet-task/badge.svg?branch=master\" alt=\"Coverage Status\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/XadillaX/scarlet-task/actions\"\u003e\n    \u003cimg src=\"https://github.com/XadillaX/scarlet-task/workflows/Node.js%20CI/badge.svg\" alt=\"Build Status\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\nScarlet Task is an advanced task queue module for Node.js, featuring the ability to configure multiple child queues for\na single task queue.\n\n## Why named Scarlet? ๛ก(ｰ̀ωｰ́ก)\n\nThe story behind this module's name is quite serendipitous. One day, while browsing [萌否收音機](https://moe.fm), I\nstumbled upon a captivating song. Though its title slipped my mind, I distinctly remembered its considerable length.\n\nDriven by a desire to rediscover this elusive track, I developed this queue module. It allowed me to perform an\nexhaustive search across the entire Moe FM platform. After a thorough exploration, I finally unearthed the song that had\nbeen haunting my memory: \u003c[the Embodiment of Scarlet Devil](https://moe.fm/listen?song=79922)\u003e.\n\nTo commemorate this musical treasure hunt and pay homage to my favorite character,\n[Flandre Scarlet](http://touhou.wikia.com/wiki/Flandre_Scarlet), I christened this module 'Scarlet Task'.\nIt's a nod to both the journey of rediscovery and the Scarlet-themed song that sparked it all.\n\n## Use Case\n\nScarlet Task is particularly useful for scenarios requiring controlled asynchronous operations, such as web crawling. By\nallowing you to set up multiple child queues, it enables concurrent processing while maintaining control over\nparallelism, preventing issues like unintentional DDoS-like behavior on target websites.\n\n## Installation\n\n```shell\n$ npm install --save scarlet-task\n```\n\nIf you're using it in browser, you may need to install `events` as well.\n\n```shell\n$ npm install --save events\n```\n\n## Tutorials\n\nFirst, require the module and instantiate a Scarlet object:\n\n```javascript\nconst { Scarlet } = require(\"scarlet-task\");\nconst scarlet = new Scarlet(10);\n```\n\n\u003e The parameter for the `constructor` represents the number of child queues. If no parameter is passed, it defaults to 1 child queue.\n\nDefine a `processor` function to handle tasks. You can also pass an anonymous function:\n\n```javascript\nfunction processor(taskObject) {\n  // Get the task object\n  const task = taskObject.task;\n  \n  // Perform task logic...\n  // ...\n  \n  taskObject.done(); // Or call `scarlet.taskDone(taskObject);`\n\n  console.log(scarlet.numberOfProcessed());\n}\n```\n\n\u003e **Note:** In the `processor` function, you should call `taskObject.done()` or `scarlet.taskDone(taskObject)` when you consider the task complete. Then `scarlet` will process the next task. The `taskObject` parameter is passed to you by the `scarlet` instance.\n\nYou can push tasks into the queue at any time. The task object can be of any type - string, number, JSON, etc.:\n\n```javascript\nconst task = \"This could be a URL, or an object that the processor can handle\";\nscarlet.push(task, processor);\n```\n\nFor more reference, see `test/hackernews.test.ts`.\n\nYou can reset the number of processed tasks:\n\n```javascript\nscarlet.resetNumberOfProcessed();\n```\n\nYou can also set an after-finish callback function that Scarlet will call after a specified number of tasks are completed:\n\n```javascript\nscarlet.afterFinish(20, done, false);\n// This will call done() after 20 tasks are completed, without looping (meaning it executes only once unless you reset the processed count)\n\nscarlet.clearAfterFinish();\n// You can clear the afterFinish processor\n```\n\n\u003e For more reference, see `test/hackernews.test.ts`.\n\n## APIs\n\n### `Scarlet`\n\n#### Constructor\n\n```javascript\nconst scarlet = new Scarlet(queueCount);\n```\n\n- `queueCount` (optional): Number of child queues. Defaults to 1 if not specified.\n\n#### Methods\n\n- `push(task, processor)`: Adds a new task to the queue.\n  - `task`: The task to be processed (can be any type).\n  - `processor`: Function to process the task.\n\n- `numberOfProcessed()`: Returns the number of processed tasks.\n\n- `resetNumberOfProcessed()`: Resets the count of processed tasks to 0.\n\n- `taskDone(taskObject)`: Marks a task as completed.\n  - `taskObject`: The task object to be marked as done.\n\n- `afterFinish(count, processor, loop)`: Sets a callback to be executed after a certain number of tasks are completed.\n  - `count`: Number of tasks to complete before calling the processor.\n  - `processor`: Function to be called after `count` tasks are completed.\n  - `loop` (optional): If true, the processor will be called repeatedly every `count` tasks. Defaults to false.\n\n- `clearAfterFinish()`: Clears the after-finish processor.\n\n### `TaskObject`\n\n#### Properties\n\n- `task`: The task data.\n- `queueId`: The ID of the queue processing this task.\n\n#### Methods\n\n- `done()`: Marks the task as completed.\n\n## Contribute\n\nYou're welcome to make pull requests!\n\n「雖然我覺得不怎麼可能有人會關注我」\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxadillax%2Fscarlet-task","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxadillax%2Fscarlet-task","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxadillax%2Fscarlet-task/lists"}