{"id":19635496,"url":"https://github.com/henrygd/queue","last_synced_at":"2025-04-04T17:05:14.328Z","repository":{"id":243589854,"uuid":"812785249","full_name":"henrygd/queue","owner":"henrygd","description":"Tiny async queue with concurrency control. Like p-limit or fastq, but smaller and faster.","archived":false,"fork":false,"pushed_at":"2024-11-19T16:36:52.000Z","size":73,"stargazers_count":92,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-28T16:07:25.712Z","etag":null,"topics":["async","async-await","asynchronous","concurrency","promise","queue"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/henrygd.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-09T21:41:22.000Z","updated_at":"2025-03-26T10:47:59.000Z","dependencies_parsed_at":"2024-06-27T00:37:50.076Z","dependency_job_id":"8a8f31d0-4941-46c4-9d85-657ac70ebb89","html_url":"https://github.com/henrygd/queue","commit_stats":null,"previous_names":["henrygd/queue"],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/henrygd%2Fqueue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/henrygd%2Fqueue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/henrygd%2Fqueue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/henrygd%2Fqueue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/henrygd","download_url":"https://codeload.github.com/henrygd/queue/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247217172,"owners_count":20903008,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async","async-await","asynchronous","concurrency","promise","queue"],"created_at":"2024-11-11T12:25:34.761Z","updated_at":"2025-04-04T17:05:14.303Z","avatar_url":"https://github.com/henrygd.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"[size-image]: https://img.shields.io/github/size/henrygd/queue/dist/index.min.js?style=flat\n[license-image]: https://img.shields.io/github/license/henrygd/queue?style=flat\u0026color=%2349ac0c\n[license-url]: /LICENSE\n\n# @henrygd/queue\n\n[![File Size][size-image]](https://github.com/henrygd/queue/blob/main/dist/index.min.js) [![MIT license][license-image]][license-url] [![JSR Score 100%](https://jsr.io/badges/@henrygd/queue/score)](https://jsr.io/@henrygd/queue)\n\nTiny async queue with concurrency control. Like `p-limit` or `fastq`, but smaller and faster. See [comparisons and benchmarks](#comparisons-and-benchmarks) below.\n\nWorks with: \u003cimg alt=\"browsers\" title=\"This package works with browsers.\" height=\"16px\" src=\"https://jsr.io/logos/browsers.svg\" /\u003e \u003cimg alt=\"Deno\" title=\"This package works with Deno.\" height=\"16px\" src=\"https://jsr.io/logos/deno.svg\" /\u003e \u003cimg alt=\"Node.js\" title=\"This package works with Node.js\" height=\"16px\" src=\"https://jsr.io/logos/node.svg\" /\u003e \u003cimg alt=\"Cloudflare Workers\" title=\"This package works with Cloudflare Workers.\" height=\"16px\" src=\"https://jsr.io/logos/cloudflare-workers.svg\" /\u003e \u003cimg alt=\"Bun\" title=\"This package works with Bun.\" height=\"16px\" src=\"https://jsr.io/logos/bun.svg\" /\u003e\n\n## Usage\n\nCreate a queue with the `newQueue` function. Then add async functions - or promise returning functions - to your queue with the `add` method.\n\nYou can use `queue.done()` to wait for the queue to be empty.\n\n\u003c!-- prettier-ignore --\u003e\n```ts\nimport { newQueue } from '@henrygd/queue'\n\n// create a new queue with a concurrency of 2\nconst queue = newQueue(2)\n\nconst pokemon = ['ditto', 'hitmonlee', 'pidgeot', 'poliwhirl', 'golem', 'charizard']\n\nfor (const name of pokemon) {\n    queue.add(async () =\u003e {\n        const res = await fetch(`https://pokeapi.co/api/v2/pokemon/${name}`)\n        const json = await res.json()\n        console.log(`${json.name}: ${json.height * 10}cm | ${json.weight / 10}kg`)\n    })\n}\n\nconsole.log('running')\nawait queue.done()\nconsole.log('done')\n```\n\nThe return value of `queue.add` is the same as the return value of the supplied function.\n\n```ts\nconst response = await queue.add(() =\u003e fetch('https://pokeapi.co/api/v2/pokemon'))\nconsole.log(response.ok, response.status, response.headers)\n```\n\n\u003e [!TIP]\n\u003e If you need support for Node's [AsyncLocalStorage](https://nodejs.org/api/async_context.html#introduction), import `@henrygd/queue/async-storage` instead.\n\n## Queue interface\n\n```ts\n/** Add an async function / promise wrapper to the queue */\nqueue.add\u003cT\u003e(promiseFunction: () =\u003e PromiseLike\u003cT\u003e): Promise\u003cT\u003e\n/** Returns a promise that resolves when the queue is empty */\nqueue.done(): Promise\u003cvoid\u003e\n/** Empties the queue (active promises are not cancelled) */\nqueue.clear(): void\n/** Returns the number of promises currently running */\nqueue.active(): number\n/** Returns the total number of promises in the queue */\nqueue.size(): number\n```\n\n## Comparisons and benchmarks\n\n| Library                                                         | Version | Bundle size (B) | Weekly downloads |\n| :-------------------------------------------------------------- | :------ | :-------------- | :--------------- |\n| @henrygd/queue                                                  | 1.0.6   | 355             | dozens :)        |\n| [p-limit](https://github.com/sindresorhus/p-limit)              | 5.0.0   | 1,763           | 118,953,973      |\n| [async.queue](https://github.com/caolan/async)                  | 3.2.5   | 6,873           | 53,645,627       |\n| [fastq](https://github.com/mcollina/fastq)                      | 1.17.1  | 3,050           | 39,257,355       |\n| [queue](https://github.com/jessetane/queue)                     | 7.0.0   | 2,840           | 4,259,101        |\n| [promise-queue](https://github.com/promise-queue/promise-queue) | 2.2.5   | 2,200           | 1,092,431        |\n\n### Note on benchmarks\n\nAll libraries run the exact same test. Each operation measures how quickly the queue can resolve 1,000 async functions. The function just increments a counter and checks if it has reached 1,000.[^benchmark]\n\nWe check for completion inside the function so that `promise-queue` and `p-limit` are not penalized by having to use `Promise.all` (they don't provide a promise that resolves when the queue is empty).\n\n## Browser benchmark\n\nThis test was run in Chromium. Chrome and Edge are the same. Firefox and Safari are slower and closer, with `@henrygd/queue` just edging out `promise-queue`. I think both are hitting the upper limit of what those browsers will allow.\n\nYou can run or tweak for yourself here: https://jsbm.dev/TKyOdie0sbpOh\n\n![@henrygd/queue - 13,665 Ops/s. fastq - 7,661 Ops/s. promise-queue - 7,650 Ops/s. async.queue - 4,060 Ops/s. p-limit - 1,067 Ops/s. queue - 721 Ops/s](https://henrygd-assets.b-cdn.net/queue/106/browser-benchmark.png)\n\n## Node.js benchmarks\n\n\u003e Note: `p-limit` 6.1.0 now places between `async.queue` and `queue` in Node and Deno.\n\nRyzen 5 4500U | 8GB RAM | Node 22.3.0\n\n![@henrygd/queue - 1.9x faster than fastq. 2.03x promise-queue. 3.86x async.queue. 20x queue. 86x p-limit.](https://henrygd-assets.b-cdn.net/queue/106/node-4500.png)\n\nRyzen 7 6800H | 32GB RAM | Node 22.3.0\n\n![@henrygd/queue - 1.9x faster than fastq. 2.01x promise-queue. 3.98x async.queue. 6.86x queue. 88x p-limit.](https://henrygd-assets.b-cdn.net/queue/106/node-6800h.png)\n\n## Deno benchmarks\n\n\u003e Note: `p-limit` 6.1.0 now places between `async.queue` and `queue` in Node and Deno.\n\nRyzen 5 4500U | 8GB RAM | Deno 1.44.4\n\n![@henrygd/queue - 1.9x faster than fastq. 2.01x promise-queue. 4.7x async.queue. 7x queue. 28x p-limit.](https://henrygd-assets.b-cdn.net/queue/106/deno-4500.png)\n\nRyzen 7 6800H | 32GB RAM | Deno 1.44.4\n\n![@henrygd/queue - 1.82x faster than fastq. 1.91x promise-queue. 3.47x async.queue. 7x queue. 26x p-limit.](https://henrygd-assets.b-cdn.net/queue/106/deno-6800h.png)\n\n## Bun benchmarks\n\nRyzen 5 4500U | 8GB RAM | Bun 1.1.17\n\n![@henrygd/queue - 1.25x faster than promise-queue. 1.66x fastq. 2.73x async.queue. 5.44x p-limit. 12x queue.](https://henrygd-assets.b-cdn.net/queue/106/bun-4500.png)\n\nRyzen 7 6800H | 32GB RAM | Bun 1.1.17\n\n![@henrygd/queue - 1.17x faster than promise-queue. 1.51x fastq. 2.53x async.queue. 5.25x p-limit. 5.39x queue.](https://henrygd-assets.b-cdn.net/queue/106/bun-6800h.png)\n\n## Cloudflare Workers benchmark\n\nUses [oha](https://github.com/hatoo/oha) to make 1,000 requests to each worker. Each request creates a queue and resolves 5,000 functions.\n\nThis was run locally using [Wrangler](https://developers.cloudflare.com/workers/get-started/guide/) on a Ryzen 7 6800H laptop. Wrangler uses the same [workerd](https://github.com/cloudflare/workerd) runtime as workers deployed to Cloudflare, so the relative difference should be accurate. Here's the [repository for this benchmark](https://github.com/henrygd/async-queue-wrangler-benchmark).\n\n| Library        | Requests/sec | Total (sec) | Average | Slowest |\n| :------------- | :----------- | :---------- | :------ | :------ |\n| @henrygd/queue | 816.1074     | 1.2253      | 0.0602  | 0.0864  |\n| promise-queue  | 647.2809     | 1.5449      | 0.0759  | 0.1149  |\n| fastq          | 336.7031     | 3.0877      | 0.1459  | 0.2080  |\n| async.queue    | 198.9986     | 5.0252      | 0.2468  | 0.3544  |\n| queue          | 85.6483      | 11.6757     | 0.5732  | 0.7629  |\n| p-limit        | 77.7434      | 12.8628     | 0.6316  | 0.9585  |\n\n## Related\n\n[`@henrygd/semaphore`](https://github.com/henrygd/semaphore) - Fastest javascript inline semaphores and mutexes using async / await.\n\n## License\n\n[MIT license](/LICENSE)\n\n[^benchmark]: In reality, you may not be running so many jobs at once, and your jobs will take much longer to resolve. So performance will depend more on the jobs themselves.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhenrygd%2Fqueue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhenrygd%2Fqueue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhenrygd%2Fqueue/lists"}