{"id":16376669,"url":"https://github.com/udoprog/leaky-bucket","last_synced_at":"2025-04-05T20:07:55.981Z","repository":{"id":44401298,"uuid":"203132746","full_name":"udoprog/leaky-bucket","owner":"udoprog","description":"A token-based rate limiter based on the leaky bucket algorithm.","archived":false,"fork":false,"pushed_at":"2024-05-22T21:00:32.000Z","size":183,"stargazers_count":86,"open_issues_count":4,"forks_count":9,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-05-22T21:58:34.701Z","etag":null,"topics":["async","leaky-bucket","rate-limiter","rate-limiting","rust","token-bucket"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/udoprog.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-19T08:39:18.000Z","updated_at":"2024-05-29T17:26:05.822Z","dependencies_parsed_at":"2024-05-29T17:26:03.279Z","dependency_job_id":null,"html_url":"https://github.com/udoprog/leaky-bucket","commit_stats":{"total_commits":76,"total_committers":4,"mean_commits":19.0,"dds":0.07894736842105265,"last_synced_commit":"2b2182333dda9b488c9b1d88d33cab55e456def8"},"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udoprog%2Fleaky-bucket","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udoprog%2Fleaky-bucket/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udoprog%2Fleaky-bucket/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udoprog%2Fleaky-bucket/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/udoprog","download_url":"https://codeload.github.com/udoprog/leaky-bucket/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247393570,"owners_count":20931812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async","leaky-bucket","rate-limiter","rate-limiting","rust","token-bucket"],"created_at":"2024-10-11T03:25:37.919Z","updated_at":"2025-04-05T20:07:55.959Z","avatar_url":"https://github.com/udoprog.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# leaky-bucket\n\n[\u003cimg alt=\"github\" src=\"https://img.shields.io/badge/github-udoprog/leaky--bucket-8da0cb?style=for-the-badge\u0026logo=github\" height=\"20\"\u003e](https://github.com/udoprog/leaky-bucket)\n[\u003cimg alt=\"crates.io\" src=\"https://img.shields.io/crates/v/leaky-bucket.svg?style=for-the-badge\u0026color=fc8d62\u0026logo=rust\" height=\"20\"\u003e](https://crates.io/crates/leaky-bucket)\n[\u003cimg alt=\"docs.rs\" src=\"https://img.shields.io/badge/docs.rs-leaky--bucket-66c2a5?style=for-the-badge\u0026logoColor=white\u0026logo=data:image/svg+xml;base64,PHN2ZyByb2xlPSJpbWciIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgdmlld0JveD0iMCAwIDUxMiA1MTIiPjxwYXRoIGZpbGw9IiNmNWY1ZjUiIGQ9Ik00ODguNiAyNTAuMkwzOTIgMjE0VjEwNS41YzAtMTUtOS4zLTI4LjQtMjMuNC0zMy43bC0xMDAtMzcuNWMtOC4xLTMuMS0xNy4xLTMuMS0yNS4zIDBsLTEwMCAzNy41Yy0xNC4xIDUuMy0yMy40IDE4LjctMjMuNCAzMy43VjIxNGwtOTYuNiAzNi4yQzkuMyAyNTUuNSAwIDI2OC45IDAgMjgzLjlWMzk0YzAgMTMuNiA3LjcgMjYuMSAxOS45IDMyLjJsMTAwIDUwYzEwLjEgNS4xIDIyLjEgNS4xIDMyLjIgMGwxMDMuOS01MiAxMDMuOSA1MmMxMC4xIDUuMSAyMi4xIDUuMSAzMi4yIDBsMTAwLTUwYzEyLjItNi4xIDE5LjktMTguNiAxOS45LTMyLjJWMjgzLjljMC0xNS05LjMtMjguNC0yMy40LTMzLjd6TTM1OCAyMTQuOGwtODUgMzEuOXYtNjguMmw4NS0zN3Y3My4zek0xNTQgMTA0LjFsMTAyLTM4LjIgMTAyIDM4LjJ2LjZsLTEwMiA0MS40LTEwMi00MS40di0uNnptODQgMjkxLjFsLTg1IDQyLjV2LTc5LjFsODUtMzguOHY3NS40em0wLTExMmwtMTAyIDQxLjQtMTAyLTQxLjR2LS42bDEwMi0zOC4yIDEwMiAzOC4ydi42em0yNDAgMTEybC04NSA0Mi41di03OS4xbDg1LTM4Ljh2NzUuNHptMC0xMTJsLTEwMiA0MS40LTEwMi00MS40di0uNmwxMDItMzguMiAxMDIgMzguMnYuNnoiPjwvcGF0aD48L3N2Zz4K\" height=\"20\"\u003e](https://docs.rs/leaky-bucket)\n[\u003cimg alt=\"build status\" src=\"https://img.shields.io/github/actions/workflow/status/udoprog/leaky-bucket/ci.yml?branch=main\u0026style=for-the-badge\" height=\"20\"\u003e](https://github.com/udoprog/leaky-bucket/actions?query=branch%3Amain)\n\nA token-based rate limiter based on the [leaky bucket] algorithm.\n\nIf the bucket overflows and goes over its max configured capacity, the task\nthat tried to acquire the tokens will be suspended until the required number\nof tokens has been drained from the bucket.\n\nSince this crate uses timing facilities from tokio it has to be used within\na Tokio runtime with the [`time` feature] enabled.\n\nThis library has some neat features, which includes:\n\n**Not requiring a background task**. This is usually needed by token bucket\nrate limiters to drive progress. Instead, one of the waiting tasks\ntemporarily assumes the role as coordinator (called the *core*). This\nreduces the amount of tasks needing to sleep, which can be a source of\njitter for imprecise sleeping implementations and tight limiters. See below\nfor more details.\n\n**Dropped tasks** release any resources they've reserved. So that\nconstructing and cancellaing asynchronous tasks to not end up taking up wait\nslots it never uses which would be the case for cell-based rate limiters.\n\n\u003cbr\u003e\n\n## Usage\n\nThe core type is [`RateLimiter`], which allows for limiting the throughput\nof a section using its [`acquire`], [`try_acquire`], and [`acquire_one`]\nmethods.\n\nThe following is a simple example where we wrap requests through a HTTP\n`Client`, to ensure that we don't exceed a given limit:\n\n```rust\nuse leaky_bucket::RateLimiter;\n\n/// A blog client.\npub struct BlogClient {\n    limiter: RateLimiter,\n    client: Client,\n}\n\nstruct Post {\n    // ..\n}\n\nimpl BlogClient {\n    /// Get all posts from the service.\n    pub async fn get_posts(\u0026self) -\u003e Result\u003cVec\u003cPost\u003e\u003e {\n        self.request(\"posts\").await\n    }\n\n    /// Perform a request against the service, limiting requests to abide by a rate limit.\n    async fn request\u003cT\u003e(\u0026self, path: \u0026str) -\u003e Result\u003cT\u003e\n    where\n        T: DeserializeOwned\n    {\n        // Before we start sending a request, we block on acquiring one token.\n        self.limiter.acquire(1).await;\n        self.client.request::\u003cT\u003e(path).await\n    }\n}\n```\n\n\u003cbr\u003e\n\n## Implementation details\n\nEach rate limiter has two acquisition modes. A fast path and a slow path.\nThe fast path is used if the desired number of tokens are readily available,\nand simply involves decrementing the number of tokens available in the\nshared pool.\n\nIf the required number of tokens is not available, the task will be forced\nto be suspended until the next refill interval. Here one of the acquiring\ntasks will switch over to work as a *core*. This is known as *core\nswitching*.\n\n```rust\nuse leaky_bucket::RateLimiter;\nuse tokio::time::Duration;\n\nlet limiter = RateLimiter::builder()\n    .initial(10)\n    .interval(Duration::from_millis(100))\n    .build();\n\n// This is instantaneous since the rate limiter starts with 10 tokens to\n// spare.\nlimiter.acquire(10).await;\n\n// This however needs to core switch and wait for a while until the desired\n// number of tokens is available.\nlimiter.acquire(3).await;\n```\n\nThe core is responsible for sleeping for the configured interval so that\nmore tokens can be added. After which it ensures that any tasks that are\nwaiting to acquire including itself are appropriately unsuspended.\n\nOn-demand core switching is what allows this rate limiter implementation to\nwork without a coordinating background thread. But we need to ensure that\nany asynchronous tasks that uses [`RateLimiter`] must either run an\n[`acquire`] call to completion, or be *cancelled* by being dropped.\n\nIf none of these hold, the core might leak and be locked indefinitely\npreventing any future use of the rate limiter from making progress. This is\nsimilar to if you would lock an asynchronous [`Mutex`] but never drop its\nguard.\n\n\u003e You can run this example with:\n\u003e\n\u003e ```sh\n\u003e cargo run --example block_forever\n\u003e ```\n\n```rust\nuse std::future::Future;\nuse std::sync::Arc;\nuse std::task::Context;\n\nuse leaky_bucket::RateLimiter;\n\nstruct Waker;\n\nlet limiter = Arc::new(RateLimiter::builder().build());\n\nlet waker = Arc::new(Waker).into();\nlet mut cx = Context::from_waker(\u0026waker);\n\nlet mut a0 = Box::pin(limiter.acquire(1));\n// Poll once to ensure that the core task is assigned.\nassert!(a0.as_mut().poll(\u0026mut cx).is_pending());\nassert!(a0.is_core());\n\n// We leak the core task, preventing the rate limiter from making progress\n// by assigning new core tasks.\nstd::mem::forget(a0);\n\n// Awaiting acquire here would block forever.\n// limiter.acquire(1).await;\n```\n\n\u003cbr\u003e\n\n## Fairness\n\nBy default [`RateLimiter`] uses a *fair* scheduler. This ensures that the\ncore task makes progress even if there are many tasks waiting to acquire\ntokens. This might cause more core switching, increasing the total work\nneeded. An unfair scheduler is expected to do a bit less work under\ncontention. But without fair scheduling some tasks might end up taking\nlonger to acquire than expected.\n\nUnfair rate limiters also have access to a fast path for acquiring tokens,\nwhich might further improve throughput.\n\nThis behavior can be tweaked with the [`Builder::fair`] option.\n\n```rust\nuse leaky_bucket::RateLimiter;\n\nlet limiter = RateLimiter::builder()\n    .fair(false)\n    .build();\n```\n\nThe `unfair-scheduling` example can showcase this phenomenon.\n\n```sh\ncargo run --example unfair_scheduling\n```\n\n```text\n# fair\nMax: 1011ms, Total: 1012ms\nTimings:\n 0: 101ms\n 1: 101ms\n 2: 101ms\n 3: 101ms\n 4: 101ms\n ...\n# unfair\nMax: 1014ms, Total: 1014ms\nTimings:\n 0: 1014ms\n 1: 101ms\n 2: 101ms\n 3: 101ms\n 4: 101ms\n ...\n```\n\nAs can be seen above the first task in the *unfair* scheduler takes longer\nto run because it prioritises releasing other tasks waiting to acquire over\nitself.\n\n[`acquire_one`]: https://docs.rs/leaky-bucket/1/leaky_bucket/struct.RateLimiter.html#method.acquire_one\n[`acquire`]: https://docs.rs/leaky-bucket/1/leaky_bucket/struct.RateLimiter.html#method.acquire\n[`Builder::fair`]: https://docs.rs/leaky-bucket/1/leaky_bucket/struct.Builder.html#method.fair\n[`Mutex`]: https://docs.rs/tokio/1/tokio/sync/struct.Mutex.html\n[`RateLimiter`]: https://docs.rs/leaky-bucket/1/leaky_bucket/struct.RateLimiter.html\n[`time` feature]: https://docs.rs/tokio/1/tokio/#feature-flags\n[`try_acquire`]: https://docs.rs/leaky-bucket/1/leaky_bucket/struct.RateLimiter.html#method.try_acquire\n[leaky bucket]: https://en.wikipedia.org/wiki/Leaky_bucket\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fudoprog%2Fleaky-bucket","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fudoprog%2Fleaky-bucket","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fudoprog%2Fleaky-bucket/lists"}