{"id":15663355,"url":"https://github.com/stevenvachon/limited-request-queue","last_synced_at":"2025-05-05T21:21:30.456Z","repository":{"id":26758074,"uuid":"30215943","full_name":"stevenvachon/limited-request-queue","owner":"stevenvachon","description":"Interactively manage concurrency for outbound requests.","archived":false,"fork":false,"pushed_at":"2021-02-27T23:16:43.000Z","size":72,"stargazers_count":17,"open_issues_count":1,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-14T09:50:55.596Z","etag":null,"topics":["concurrency","data-structure","dos","http","nodejs","priority-queue","queue","rate-limiter","requests","url","whatwg"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stevenvachon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"license","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-02-03T00:11:07.000Z","updated_at":"2024-04-18T02:04:24.000Z","dependencies_parsed_at":"2022-08-31T04:22:49.996Z","dependency_job_id":null,"html_url":"https://github.com/stevenvachon/limited-request-queue","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevenvachon%2Flimited-request-queue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevenvachon%2Flimited-request-queue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevenvachon%2Flimited-request-queue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevenvachon%2Flimited-request-queue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stevenvachon","download_url":"https://codeload.github.com/stevenvachon/limited-request-queue/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252577116,"owners_count":21770734,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concurrency","data-structure","dos","http","nodejs","priority-queue","queue","rate-limiter","requests","url","whatwg"],"created_at":"2024-10-03T13:36:52.646Z","updated_at":"2025-05-05T21:21:30.438Z","avatar_url":"https://github.com/stevenvachon.png","language":"JavaScript","readme":"# limited-request-queue [![NPM Version][npm-image]][npm-url] ![File Size][filesize-image] [![Build Status][travis-image]][travis-url] [![Coverage Status][coveralls-image]][coveralls-url] [![Dependency Monitor][greenkeeper-image]][greenkeeper-url]\n\n\u003e Interactively manage concurrency for outbound requests.\n\n\n* Concurrency \u0026 rate limiting prevents overload on your network.\n* Per-Host concurrency limiting prevents overload on your target network(s).\n* Pause/Resume at any time.\n\n```js\nconst queue = new RequestQueue()\n  .on(ITEM_EVENT, (url, data, done) =\u003e {\n    yourRequestLib(url, () =\u003e done());\n  })\n  .on(END_EVENT, () =\u003e console.log('Queue completed!'));\n\nconst urls = ['http://domain.com/dir1/', 'http://domain.com/dir2/'];\nurls.forEach(url =\u003e queue.enqueue(new URL(url)));\n\nsetTimeout(queue.pause, 500);\nsetTimeout(queue.resume, 5000);\n```\n\n\n## Installation\n\n[Node.js](https://nodejs.org) `\u003e= 14` is required. To install, type this at the command line:\n```shell\nnpm install limited-request-queue\n```\n\n\n## Usage\n\nImport as an ES Module:\n```js\nimport RequestQueue, {END_EVENT, ITEM_EVENT} from 'limited-request-queue';\n```\n\nImport as a CommonJS Module:\n```js\nconst {END_EVENT, ITEM_EVENT, RequestQueue} = require('limited-request-queue');\n```\n\nConstructor:\n```js\nnew RequestQueue(options);\n```\n\n\n## Methods \u0026 Properties\n\nAll methods from [`EventEmitter`](https://nodejs.org/api/events.html#events_class_eventemitter) are available.\n\n### `.dequeue(id)`\nRemoves a queue item from the queue. Returns `true` if a queue item was removed and `false` if not. Use of this function is likely not needed as items are auto-dequeued when their turn is reached.\n\n### `.enqueue(url[, data, options])`\nAdds a URL to the queue. Returns a queue item ID on success.\n\n* `url` *must* a [`URL`](https://mdn.io/URL/) instance.\n* `data` is optional and can be of any type.\n* `options` is an optional `Object` that overrides any defined options in the constructor (except for `maxSockets`).\n\n### `.has(id)`\nReturns `true` if the queue contains an active or queued item tagged with `id` and `false` if not.\n\n### `.isPaused`\nReturns `true` if the queue is currently paused and `false` if not.\n\n### `.length`\nReturns the total number of items in the queue, active and inactive.\n\n### `.numActive`\nReturns the number of items whose requests are currently in progress.\n\n### `.numQueued`\nReturns the number of items that have not yet made requests.\n\n### `.pause()`\nPauses the queue, but will not pause any active requests.\n\n### `.resume()`\nResumes the queue.\n\n\n## Options\n\n### `options.ignorePorts`\nType: `Boolean`  \nDefault value: `true`  \nWhether or not to treat identical hosts of different ports as a single concurrent group. **Example:** when `true`, http://mydomain.com:80 and http://mydomain.com:8080 may not have outgoing connections at the same time, but http://mydomain.com:80 and http://yourdomain.com:8080 will.\n\n### `options.ignoreProtocols`\nType: `Boolean`  \nDefault value: `true`  \nWhether or not to treat identical hosts of different protocols as a single concurrent group. **Example:** when `true`, http://mydomain.com and https://mydomain.com may not have outgoing connections at the same time, but http://mydomain.com and https://yourdomain.com will.\n\n### `options.ignoreSubdomains`\nType: `Boolean`  \nDefault value: `true`  \nWhether or not to treat identical domains of different subdomains as a single concurrent group. **Example:** when `true`, http://mydomain.com and http://www.mydomain.com may not have outgoing connections at the same time, but http://mydomain.com and http://www.yourdomain.com will.\n\nThis option is not available in the browser version (due to extreme file size).\n\n### `options.maxSockets`\nType: `Number`  \nDefault value: `Infinity`  \nThe maximum number of connections allowed at any given time. A value of `0` will prevent anything from going out. A value of `Infinity` will provide no concurrency limiting.\n\n### `options.maxSocketsPerHost`\nType: `Number`  \nDefault value: `2`  \nThe maximum number of connections per host allowed at any given time. A value of `0` will prevent anything from going out. A value of `Infinity` will provide no per-host concurrency limiting.\n\n### `options.rateLimit`\nType: `Number`  \nDefault value: `0`  \nThe number of milliseconds to wait before each request. For a typical rate limiter, also set `maxSockets` to `1`.\n\n\n## Events\n\n### `END_EVENT`, `'end'`\nCalled when the last item in the queue has been completed/dequeued.\n\n### `ITEM_EVENT`, `'item'`\nCalled when a queue item's turn has been reached. Arguments are: `url`, `data`, `done`. Call the `done` function when your item's operations are complete.\n\n\n[npm-image]: https://img.shields.io/npm/v/limited-request-queue.svg\n[npm-url]: https://npmjs.org/package/limited-request-queue\n[filesize-image]: https://img.shields.io/badge/bundle-40kB%20gzipped-blue.svg\n[travis-image]: https://img.shields.io/travis/stevenvachon/limited-request-queue.svg\n[travis-url]: https://travis-ci.org/stevenvachon/limited-request-queue\n[coveralls-image]: https://img.shields.io/coveralls/stevenvachon/limited-request-queue.svg\n[coveralls-url]: https://coveralls.io/github/stevenvachon/limited-request-queue\n[greenkeeper-image]: https://badges.greenkeeper.io/stevenvachon/limited-request-queue.svg\n[greenkeeper-url]: https://greenkeeper.io/\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstevenvachon%2Flimited-request-queue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstevenvachon%2Flimited-request-queue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstevenvachon%2Flimited-request-queue/lists"}