{"id":25996919,"url":"https://github.com/clickup/distributed-pacer","last_synced_at":"2025-07-10T05:39:47.038Z","repository":{"id":254993318,"uuid":"846287141","full_name":"clickup/distributed-pacer","owner":"clickup","description":"A concurrency aware Redis-backed rate limiter with pacing delay prediction and Token Bucket bursts handling","archived":false,"fork":false,"pushed_at":"2025-02-20T16:24:00.000Z","size":84,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-13T03:49:43.955Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/@clickup/distributed-pacer","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/clickup.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-22T22:35:14.000Z","updated_at":"2025-02-20T16:18:26.000Z","dependencies_parsed_at":null,"dependency_job_id":"92ed2673-3a39-4468-a1d5-696c57faf1c0","html_url":"https://github.com/clickup/distributed-pacer","commit_stats":null,"previous_names":["clickup/distributed-pacer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/clickup/distributed-pacer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clickup%2Fdistributed-pacer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clickup%2Fdistributed-pacer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clickup%2Fdistributed-pacer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clickup%2Fdistributed-pacer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clickup","download_url":"https://codeload.github.com/clickup/distributed-pacer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clickup%2Fdistributed-pacer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264536025,"owners_count":23624405,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-05T16:55:38.773Z","updated_at":"2025-07-10T05:39:47.017Z","avatar_url":"https://github.com/clickup.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# distributed-pacer: A concurrency aware Redis-backed rate limiter with pacing delay prediction and Token Bucket bursts handling\n\nSee also [Full API documentation](https://github.com/clickup/distributed-pacer/blob/master/docs/modules.md).\n\n![CI run](https://github.com/clickup/distributed-pacer/actions/workflows/ci.yml/badge.svg?branch=main)\n\n## Pacing\n\nPacing controls the rate at which concurrent clients perform some operations. It\nintroduces deliberate delays between client requests. The primary goal of pacing\nis to ensure that the rate of operations (such as outgoing requests) does not\nexceed a certain threshold (e.g. QPS - \"queries per second\").\n\nUse Cases:\n\n- **Outgoing Requests.** Pacing is typically used by clients to manage the rate\n  at which they send requests to an external service that imposes rate limits.\n- **Load Management.** In scenarios where the external service might be\n  sensitive to sudden spikes in traffic, pacing helps in distributing the load\n  more evenly over time.\n\nNotice that the term \"pacing\" is typically used for outgoing requests from\nclients (to slow down the requests flow without dropping them), whilst \"rate\nlimiting\" is for incoming requests on servers (to reject non-conforming\nrequests).\n\n### Pacing Usage Example\n\n```ts\nimport { DistributedPacer } from \"@clickup/distributed-pacer\";\nimport { Redis } from \"ioredis\";\nimport { setTimeout } from \"timers/promises\";\n\nconst myIoRedis = new Redis();\n\nasync function mySendApiRequest() {\n  // sends an API request somewhere\n}\n\nasync function myWorkerRunningOnMultipleMachines() {\n  while (true) {\n    const lightweightPacer = new DistributedPacer(myIoRedis, {\n      key: \"myKey\",\n      qps: 10,\n      maxBurst: 1, // optional\n      burstAllowanceFactor: 0.5, // optional\n    });\n    const outcome = await lightweightPacer.pace(1 /* weight */);\n\n    console.log(outcome.reason);\n    await setTimeout(outcome.delayMs);\n\n    await mySendApiRequest();\n  }\n}\n```\n\n### How it Works\n\n`DistributedPacer` spreads the requests issued by some concurrent workers or\nprocesses uniformly into the future to satisfy the desired downstream QPS\n(queries per second) exactly. The implementation is inspired by Leaky Bucket for\nQueues algorithm.\n\nThe general use case is to introduce some artificial back-pressure when sending\nrequests to external services, to avoid overloading them, e.g.:\n\n- Pacing outgoing requests to some external API to meet its rate limits.\n- Protecting the local database from overloading with concurrent writes done by\n  multiple workers.\n\nImagine we have a time machine, and we can send requests (events) into the exact\nprovided moment of time in the future. To send a request into the future, the\nLua script in Redis returns that moment's timestamp, and then the worker needs\nto call delay() to wake up at that moment. We also store the last moment of the\nfuture to where we sent a previous request, so next requests coming (if they\ncome too quickly) will be sent further and further away.\n\nAnother analogy is booking a meeting in the calendar. When a new request\narrives, it's not executed immediately, but instead scheduled in the calendar\naccording to the QPS allowance.\n\nThus, after the returned `delayMs` is awaited, the request will happen in at\nleast 1/QPS seconds after the previous request; thus, it will satisfy the target\nQPS. Also, if there were no requests in the past within 1/QPS seconds from the\npresent time, then `delayMs` returned will be 0.\n\n### Bursts Allowance\n\nImagine that each call to `pace(weight)` adds `weight` of water to the bucket of\n`maxBurst` volume, and every second, `1/qps*burstAllowanceFactor` of water leaks\nout of the bucket at a constant rate (but only when the pacer is idle, i.e.\nthere are no requests scheduled to the future on top of the bucket). If the\nbucket is not yet full (its watermark is below `maxBurst` level), then the\nreturned `delayMs` will be 0, so the worker can proceed with the request\nimmediately. Otherwise, pacing will start to happen. I.e. we pace only the\nrequests which cause the bucket to overflow (Leaky Bucket algorithm).\n\nThe default value of `burstAllowanceFactor` is less than 1, which forces the\nburst allowance to be earned slightly slower than the target QPS.\n\n## Rate Limiting\n\nAlthough pacing is the primary use case for this module, it also supports \"rate\nlimiting\" mode, where it's expected that requests out of quota will be rejected\n(instead of being delayed). This is useful for handling *incoming* requests on\nservers (as opposed to pacing, where the requests *originate* from workers).\n\nTo use `DistributedPacer` in rate limiting mode, call `rateLimit()` method on\nit. Logically, it works exactly the same way as `pace()`, but when it returns a\nnon-zero `delayMs`, it doesn't alter the state in Redis, assuming that the\nrequest will be rejected and won't contribute to `maxBurst` allowance.\n\nTo utilize the power of Leaky Bucket algorithm (or its equivalent here, Token\nBucket), pass a nonzero value to `maxBurst`. With the default value (which is\n0), no bursts will be allowed, so the requests will need to come in not less\nthan `1/qps` seconds in between.\n\n### Rate Limiting Usage Example\n\nDisclaimer: here, we use Express just as an illustration: there is obviously a\nready middleware module for Express rate limiting use case. Use\n`DistributedPacer` in other applications, like GraphQL processing, WebSockets,\ninternal IO services etc.\n\n```ts\nimport { DistributedPacer } from \"@clickup/distributed-pacer\";\nimport { Redis } from \"ioredis\";\nimport express from \"express\";\n\nconst myIoRedis = new Redis();\n\nexpress()\n  .get('/', (req, res) =\u003e {\n    const lightweightPacer = new DistributedPacer(myIoRedis, {\n      key: \"myKey\",\n      qps: 10,\n      maxBurst: 20,\n    });\n    const outcome = await lightweightPacer.rateLimit(1 /* weight */);\n\n    if (outcome.delay \u003e 0) {\n      console.log(outcome.reason);\n      res.status(429).send(`Rate limited, try again in ${outcome.delay} ms.`);\n    } else {\n      res.send(\"Hello World!\")\n    }\n  })\n  .listen(port);\n```\n\n## Performance\n\nThis module is cheap and can be put on a critical path in your application.\n\n`DistributedPacer` objects are lightweight, so you can create them as often as\nyou want (even on every request).\n\nEach call to `pace()` or `rateLimit()` causes one round-trip to Redis (it runs a\ncustom Lua function), and the timing of that call is O(1).\n\nIf you need to use multiple keys, you can use Redis in cluster mode to spread\nthose keys across multiple Redis nodes (pass an instance of `Redis.Cluster` to\n`DistributedPacer` constructor).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclickup%2Fdistributed-pacer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclickup%2Fdistributed-pacer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclickup%2Fdistributed-pacer/lists"}