{"id":13464781,"url":"https://github.com/mcollina/fastq","last_synced_at":"2025-05-11T05:46:32.138Z","repository":{"id":33714609,"uuid":"37368217","full_name":"mcollina/fastq","owner":"mcollina","description":"Fast, in memory work queue","archived":false,"fork":false,"pushed_at":"2025-03-20T10:42:10.000Z","size":82,"stargazers_count":1056,"open_issues_count":17,"forks_count":53,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-05-11T05:46:26.935Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"isc","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mcollina.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["mcollina"]}},"created_at":"2015-06-13T10:57:43.000Z","updated_at":"2025-05-10T22:41:41.000Z","dependencies_parsed_at":"2023-01-15T02:11:04.573Z","dependency_job_id":"f8583bab-c0d0-4ebb-83e7-cc08b21e96d5","html_url":"https://github.com/mcollina/fastq","commit_stats":{"total_commits":100,"total_committers":27,"mean_commits":"3.7037037037037037","dds":0.47,"last_synced_commit":"f06ebb8ac13c2f2a29b4a03fbaffb09e0eaaadc9"},"previous_names":[],"tags_count":26,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcollina%2Ffastq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcollina%2Ffastq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcollina%2Ffastq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcollina%2Ffastq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mcollina","download_url":"https://codeload.github.com/mcollina/fastq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253523718,"owners_count":21921818,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T14:00:50.233Z","updated_at":"2025-05-11T05:46:32.120Z","avatar_url":"https://github.com/mcollina.png","language":"JavaScript","funding_links":["https://github.com/sponsors/mcollina"],"categories":["JavaScript","Packages"],"sub_categories":[],"readme":"# fastq\n\n![ci][ci-url]\n[![npm version][npm-badge]][npm-url]\n\nFast, in memory work queue.\n\nBenchmarks (1 million tasks):\n\n* setImmediate: 812ms\n* fastq: 854ms\n* async.queue: 1298ms\n* neoAsync.queue: 1249ms\n\nObtained on node 12.16.1, on a dedicated server.\n\nIf you need zero-overhead series function call, check out\n[fastseries](http://npm.im/fastseries). For zero-overhead parallel\nfunction call, check out [fastparallel](http://npm.im/fastparallel).\n\n[![js-standard-style](https://raw.githubusercontent.com/feross/standard/master/badge.png)](https://github.com/feross/standard)\n\n  * \u003ca href=\"#install\"\u003eInstallation\u003c/a\u003e\n  * \u003ca href=\"#usage\"\u003eUsage\u003c/a\u003e\n  * \u003ca href=\"#api\"\u003eAPI\u003c/a\u003e\n  * \u003ca href=\"#license\"\u003eLicence \u0026amp; copyright\u003c/a\u003e\n\n## Install\n\n`npm i fastq --save`\n\n## Usage (callback API)\n\n```js\n'use strict'\n\nconst queue = require('fastq')(worker, 1)\n\nqueue.push(42, function (err, result) {\n  if (err) { throw err }\n  console.log('the result is', result)\n})\n\nfunction worker (arg, cb) {\n  cb(null, arg * 2)\n}\n```\n\n## Usage (promise API)\n\n```js\nconst queue = require('fastq').promise(worker, 1)\n\nasync function worker (arg) {\n  return arg * 2\n}\n\nasync function run () {\n  const result = await queue.push(42)\n  console.log('the result is', result)\n}\n\nrun()\n```\n\n### Setting \"this\"\n\n```js\n'use strict'\n\nconst that = { hello: 'world' }\nconst queue = require('fastq')(that, worker, 1)\n\nqueue.push(42, function (err, result) {\n  if (err) { throw err }\n  console.log(this)\n  console.log('the result is', result)\n})\n\nfunction worker (arg, cb) {\n  console.log(this)\n  cb(null, arg * 2)\n}\n```\n\n### Using with TypeScript (callback API)\n\n```ts\n'use strict'\n\nimport * as fastq from \"fastq\";\nimport type { queue, done } from \"fastq\";\n\ntype Task = {\n  id: number\n}\n\nconst q: queue\u003cTask\u003e = fastq(worker, 1)\n\nq.push({ id: 42})\n\nfunction worker (arg: Task, cb: done) {\n  console.log(arg.id)\n  cb(null)\n}\n```\n\n### Using with TypeScript (promise API)\n\n```ts\n'use strict'\n\nimport * as fastq from \"fastq\";\nimport type { queueAsPromised } from \"fastq\";\n\ntype Task = {\n  id: number\n}\n\nconst q: queueAsPromised\u003cTask\u003e = fastq.promise(asyncWorker, 1)\n\nq.push({ id: 42}).catch((err) =\u003e console.error(err))\n\nasync function asyncWorker (arg: Task): Promise\u003cvoid\u003e {\n  // No need for a try-catch block, fastq handles errors automatically\n  console.log(arg.id)\n}\n```\n\n## API\n\n* \u003ca href=\"#fastqueue\"\u003e\u003ccode\u003efastqueue()\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#push\"\u003e\u003ccode\u003equeue#\u003cb\u003epush()\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#unshift\"\u003e\u003ccode\u003equeue#\u003cb\u003eunshift()\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#pause\"\u003e\u003ccode\u003equeue#\u003cb\u003epause()\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#resume\"\u003e\u003ccode\u003equeue#\u003cb\u003eresume()\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#idle\"\u003e\u003ccode\u003equeue#\u003cb\u003eidle()\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#length\"\u003e\u003ccode\u003equeue#\u003cb\u003elength()\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#getQueue\"\u003e\u003ccode\u003equeue#\u003cb\u003egetQueue()\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#kill\"\u003e\u003ccode\u003equeue#\u003cb\u003ekill()\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#killAndDrain\"\u003e\u003ccode\u003equeue#\u003cb\u003ekillAndDrain()\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#error\"\u003e\u003ccode\u003equeue#\u003cb\u003eerror()\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#concurrency\"\u003e\u003ccode\u003equeue#\u003cb\u003econcurrency\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#drain\"\u003e\u003ccode\u003equeue#\u003cb\u003edrain\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#empty\"\u003e\u003ccode\u003equeue#\u003cb\u003eempty\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#saturated\"\u003e\u003ccode\u003equeue#\u003cb\u003esaturated\u003c/b\u003e\u003c/code\u003e\u003c/a\u003e\n* \u003ca href=\"#promise\"\u003e\u003ccode\u003efastqueue.promise()\u003c/code\u003e\u003c/a\u003e\n\n-------------------------------------------------------\n\u003ca name=\"fastqueue\"\u003e\u003c/a\u003e\n### fastqueue([that], worker, concurrency)\n\nCreates a new queue.\n\nArguments:\n\n* `that`, optional context of the `worker` function.\n* `worker`, worker function, it would be called with `that` as `this`,\n  if that is specified.\n* `concurrency`, number of concurrent tasks that could be executed in\n  parallel.\n\n-------------------------------------------------------\n\u003ca name=\"push\"\u003e\u003c/a\u003e\n### queue.push(task, done)\n\nAdd a task at the end of the queue. `done(err, result)` will be called\nwhen the task was processed.\n\n-------------------------------------------------------\n\u003ca name=\"unshift\"\u003e\u003c/a\u003e\n### queue.unshift(task, done)\n\nAdd a task at the beginning of the queue. `done(err, result)` will be called\nwhen the task was processed.\n\n-------------------------------------------------------\n\u003ca name=\"pause\"\u003e\u003c/a\u003e\n### queue.pause()\n\nPause the processing of tasks. Currently worked tasks are not\nstopped.\n\n-------------------------------------------------------\n\u003ca name=\"resume\"\u003e\u003c/a\u003e\n### queue.resume()\n\nResume the processing of tasks.\n\n-------------------------------------------------------\n\u003ca name=\"idle\"\u003e\u003c/a\u003e\n### queue.idle()\n\nReturns `false` if there are tasks being processed or waiting to be processed.\n`true` otherwise.\n\n-------------------------------------------------------\n\u003ca name=\"length\"\u003e\u003c/a\u003e\n### queue.length()\n\nReturns the number of tasks waiting to be processed (in the queue).\n\n-------------------------------------------------------\n\u003ca name=\"getQueue\"\u003e\u003c/a\u003e\n### queue.getQueue()\n\nReturns all the tasks be processed (in the queue). Returns empty array when there are no tasks\n\n-------------------------------------------------------\n\u003ca name=\"kill\"\u003e\u003c/a\u003e\n### queue.kill()\n\nRemoves all tasks waiting to be processed, and reset `drain` to an empty\nfunction.\n\n-------------------------------------------------------\n\u003ca name=\"killAndDrain\"\u003e\u003c/a\u003e\n### queue.killAndDrain()\n\nSame than `kill` but the `drain` function will be called before reset to empty.\n\n-------------------------------------------------------\n\u003ca name=\"error\"\u003e\u003c/a\u003e\n### queue.error(handler)\n\nSet a global error handler. `handler(err, task)` will be called\neach time a task is completed, `err` will be not null if the task has thrown an error.\n\n-------------------------------------------------------\n\u003ca name=\"concurrency\"\u003e\u003c/a\u003e\n### queue.concurrency\n\nProperty that returns the number of concurrent tasks that could be executed in\nparallel. It can be altered at runtime.\n\n-------------------------------------------------------\n\u003ca name=\"paused\"\u003e\u003c/a\u003e\n### queue.paused\n\nProperty (Read-Only) that returns `true` when the queue is in a paused state.\n\n-------------------------------------------------------\n\u003ca name=\"drain\"\u003e\u003c/a\u003e\n### queue.drain\n\nFunction that will be called when the last\nitem from the queue has been processed by a worker.\nIt can be altered at runtime.\n\n-------------------------------------------------------\n\u003ca name=\"empty\"\u003e\u003c/a\u003e\n### queue.empty\n\nFunction that will be called when the last\nitem from the queue has been assigned to a worker.\nIt can be altered at runtime.\n\n-------------------------------------------------------\n\u003ca name=\"saturated\"\u003e\u003c/a\u003e\n### queue.saturated\n\nFunction that will be called when the queue hits the concurrency\nlimit.\nIt can be altered at runtime.\n\n-------------------------------------------------------\n\u003ca name=\"promise\"\u003e\u003c/a\u003e\n### fastqueue.promise([that], worker(arg), concurrency)\n\nCreates a new queue with `Promise` apis. It also offers all the methods\nand properties of the object returned by [`fastqueue`](#fastqueue) with the modified\n[`push`](#pushPromise) and [`unshift`](#unshiftPromise) methods.\n\nNode v10+ is required to use the promisified version.\n\nArguments:\n* `that`, optional context of the `worker` function.\n* `worker`, worker function, it would be called with `that` as `this`,\n  if that is specified. It MUST return a `Promise`.\n* `concurrency`, number of concurrent tasks that could be executed in\n  parallel.\n\n\u003ca name=\"pushPromise\"\u003e\u003c/a\u003e\n#### queue.push(task) =\u003e Promise\n\nAdd a task at the end of the queue. The returned `Promise`  will be fulfilled (rejected)\nwhen the task is completed successfully (unsuccessfully).\n\nThis promise could be ignored as it will not lead to a `'unhandledRejection'`.\n\n\u003ca name=\"unshiftPromise\"\u003e\u003c/a\u003e\n#### queue.unshift(task) =\u003e Promise\n\nAdd a task at the beginning of the queue. The returned `Promise`  will be fulfilled (rejected)\nwhen the task is completed successfully (unsuccessfully).\n\nThis promise could be ignored as it will not lead to a `'unhandledRejection'`.\n\n\u003ca name=\"drained\"\u003e\u003c/a\u003e\n#### queue.drained() =\u003e Promise\n\nWait for the queue to be drained. The returned `Promise` will be resolved when all tasks in the queue have been processed by a worker.\n\nThis promise could be ignored as it will not lead to a `'unhandledRejection'`.\n\n## License\n\nISC\n\n[ci-url]: https://github.com/mcollina/fastq/workflows/ci/badge.svg\n[npm-badge]: https://badge.fury.io/js/fastq.svg\n[npm-url]: https://badge.fury.io/js/fastq\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcollina%2Ffastq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmcollina%2Ffastq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcollina%2Ffastq/lists"}