{"id":23114378,"url":"https://github.com/snok/self-limiters","last_synced_at":"2025-08-16T20:32:01.087Z","repository":{"id":59265933,"uuid":"522294201","full_name":"snok/self-limiters","owner":"snok","description":"Async distributed rate limiters for Python","archived":false,"fork":false,"pushed_at":"2024-01-18T05:05:26.000Z","size":2329,"stargazers_count":29,"open_issues_count":2,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-11T16:37:20.472Z","etag":null,"topics":["async","asyncio","distributed","python","rate-limiter","redis","rust","semaphore","tokenbucket"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-4-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/snok.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-08-07T18:34:37.000Z","updated_at":"2024-02-08T15:36:45.000Z","dependencies_parsed_at":"2023-12-07T11:30:50.030Z","dependency_job_id":"dd96350a-f7d0-49da-86d3-1cb89fb5fe53","html_url":"https://github.com/snok/self-limiters","commit_stats":null,"previous_names":["sondrelg/self-limiters"],"tags_count":26,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snok%2Fself-limiters","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snok%2Fself-limiters/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snok%2Fself-limiters/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snok%2Fself-limiters/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/snok","download_url":"https://codeload.github.com/snok/self-limiters/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230057987,"owners_count":18166178,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async","asyncio","distributed","python","rate-limiter","redis","rust","semaphore","tokenbucket"],"created_at":"2024-12-17T03:29:57.425Z","updated_at":"2024-12-17T03:29:58.118Z","avatar_url":"https://github.com/snok.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ca href=\"https://pypi.org/project/self-limiters/\"\u003e\u003cimg alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/self-limiters.svg\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/snok/self-limiters/actions/workflows/publish.yml\"\u003e\u003cimg alt=\"test status\" src=\"https://github.com/snok/self-limiters/actions/workflows/publish.yml/badge.svg\"\u003e\u003c/a\u003e\n\u003ca href=\"https://codecov.io/gh/sondrelg/self-limiters/\"\u003e\u003cimg alt=\"coverage\" src=\"https://codecov.io/gh/sondrelg/self-limiters/branch/main/graph/badge.svg?token=Q4YJPOFC1F\"\u003e\u003c/a\u003e\n\u003cimg alt=\"python version\" src=\"https://img.shields.io/badge/python-3.9%2B-blue\"\u003e\n\n\u003e Project no longer maintained\n\u003e\n\u003e This was a fun experiment, but since the library is i/o-bound, the long-term maintenance burden of a Rust Python plugin did not seem worth it. Instead I decided to re-implement this in Python.\n\u003e\n\u003e See https://github.com/otovo/redis-rate-limiters for the python version :)\n\n\n# Self limiters\n\nA library for regulating traffic with respect to **concurrency** or **time**.\n\nIt implements a [semaphore](https://en.wikipedia.org/wiki/Semaphore_(programming)) to be used when you need to\nlimit the number of *concurrent* requests to an API (or other resources). For example if you can at most\nsend 5 requests at the same time.\n\nIt also implements the [token bucket algorithm](https://en.wikipedia.org/wiki/Token_bucket) which can be used\nto limit the number of requests made in a given time interval. For example if you're restricted to\nsending, at most, 10 requests per second.\n\nBoth limiters are async, FIFO, and distributed using Redis. This means the limiters can be\nused across several distributed processes. If that isn't your use-case, you should probably\nlook for another library which does this in memory.\n\nThe package was written with rate-limiting in mind, but the semaphore and token bucket\nimplementations can be used for anything.\n\n# Installation\n\n```\npip install self-limiters\n```\n\n# Usage\n\nBoth implementations are written as async context managers.\n\n### Semaphore\n\nThe `Semaphore` can be used like this:\n\n```python\nfrom self_limiters import Semaphore\n\n\n# 5 requests at the time\nasync with Semaphore(name=\"\", capacity=5, max_sleep=60, redis_url=\"\"):\n      client.get(...)\n```\n\nThe `name` given is what determines whether your processes will use the same limiter or not.\n\nThe semaphore implementation is largely a wrapper around the [`blpop`](https://redis.io/commands/blpop/)\nredis command. We use it to wait for the semaphore to be freed up, in a non-blocking way.\n\nIf you specify a non-zero `max_sleep`, a `MaxSleepExceededError` will be raised if `blpop` waits for longer than that specified value.\n\n### Token bucket\n\nThe `TokenBucket` context manager is used the same way, like this:\n\n```python\nfrom self_limiters import TokenBucket\n\n\n# 1 requests per minute\nasync with TokenBucket(\n        name=\"\",\n        capacity=1,\n        refill_amount=1,\n        refill_frequency=60,\n        max_sleep=600,\n        redis_url=\"\"\n):\n    client.get(...)\n```\n\nThe limiter first estimates when there will be capacity in the bucket - i.e., when it's this instances turn to go,\nthen async sleeps until then.\n\nIf `max_sleep` is set and the estimated sleep time exceeds this, a `MaxSleepExceededError`\nis raised immediately.\n\n### As a decorator\n\nThe package doesn't ship any decorators, but if you would\nlike to limit the rate at which a whole function is run,\nyou can create your own, like this:\n\n```python\nfrom self_limiters import Semaphore\n\n\n# Define a decorator function\ndef limit(name, capacity):\n  def middle(f):\n    async def inner(*args, **kwargs):\n      async with Semaphore(\n              name=name,\n              capacity=capacity,\n              redis_url=\"redis://127.0.0.1:6389\"\n      ):\n        return await f(*args, **kwargs)\n    return inner\n  return middle\n\n\n# Then pass the relevant limiter arguments like this\n@limit(name=\"foo\", capacity=5)\ndef fetch_foo(id: UUID) -\u003e Foo:\n```\n\n# Implementation and general flow\n\nThe library is written in Rust (for fun) and more importantly, relies on\n[Lua](http://www.lua.org/about.html) scripts and\n[pipelining](https://docs.rs/redis/0.22.0/redis/struct.Pipeline.html) to\nimprove the performance of each implementation.\n\nRedis lets users upload and execute Lua scripts on the server directly, meaning we can write\ne.g., the entire token bucket logic in Lua. Using Lua scripts presents a couple of nice benefits:\n\n- Since they are executed on the redis instance, we can make 1 request to redis\n  where we would otherwise have to make 3 or 4. The time saved by reducing\n  the number of requests can be significant.\n\n- Redis is single-threaded and guarantees atomic execution of scripts, meaning\n  we don't have to worry about data races. As a consequence, our implementations\n  become FIFO out of the box. Without atomic execution we'd need distributed locks to prevent race conditions, which would\n  be very expensive.\n\nSo Lua scripts help make our implementation faster and simpler.\n\nThis is the rough flow of execution, for each implementation:\n\n### The semaphore implementation\n\n1. Run a [lua script](https://github.com/snok/self-limiters/blob/main/scripts/semaphore.lua)\n   to create a list data structure in redis, as the foundation of the semaphore.\n\n   This script is idempotent, and skipped if it has already been created.\n\n2. Run [`BLPOP`](https://redis.io/commands/blpop/) to non-blockingly wait until the semaphore has capacity,\n   and pop from the list when it does.\n\n3. Then run a [pipelined command](https://github.com/snok/self-limiters/blob/main/src/semaphore.rs#L78:L83)\n   to release the semaphore by adding back the capacity borrowed.\n\nSo in total we make 3 calls to redis, which are all non-blocking.\n\n### The token bucket implementation\n\nThe token bucket implementation is even simpler. The steps are:\n\n1. Run a [lua script](https://github.com/snok/self-limiters/blob/main/scripts/token_bucket.lua)\n   to estimate and return a wake-up time.\n\n2. Sleep until the given timestamp.\n\nWe make one call, then sleep. Both are non-blocking.\n\n---------\n\nSo in summary, almost all of the time is spent waiting for async i/o, meaning the limiters' impact on an\napplication event-loop should be close to completely negligible.\n\n## Benchmarks\n\nWe run benchmarks in CI with Github actions. For a normal `ubuntu-latest` runner, we see runtimes for both limiters:\n\nWhen creating 100 instances of each implementation and calling them at the same time, the average runtimes are:\n\n- Semaphore implementation: ~0.6ms per instance\n- Token bucket implementation: ~0.03ms per instance\n\nTake a look at the [benchmarking script](https://github.com/snok/self-limiters/blob/main/src/bench.py) if you want\nto run your own tests!\n\n# Implementation reference\n\n## The semaphore implementation\n\nThe semaphore implementation is useful when you need to limit a process\nto `n` concurrent actions. For example if you have several web servers, and\nyou're interacting with an API that will only tolerate a certain amount of\nconcurrent requests before locking you out.\n\nThe flow can be broken down as follows:\n\n\u003cimg width=500 src=\"docs/semaphore.png\"\u003e\u003c/img\u003e\n\nThe initial [lua script](https://github.com/snok/self-limiters/blob/main/scripts/semaphore.lua)\nfirst checks if the redis list we will build the semaphore on exists or not.\nIt does this by calling [`SETNX`](https://redis.io/commands/setnx/) on the key of the queue plus a postfix\n(if the `name` specified in the class instantiation is \"my-queue\", then the queue name will be\n`__self-limiters:my-queue` and setnx will be called for `__self-limiters:my-queue-exists`). If the returned\nvalue is 1 it means the queue we will use for our semaphore does not exist yet and needs to be created.\n\nIt might strike you as weird to maintain a separate value, just to indicate whether a list exists,\nwhen we could just check the list itself. It would be nice if we could use\n[`EXISTS`](https://redis.io/commands/exists/) on the list directly, but unfortunately a list is considered\nnot to exist when all elements are popped (i.e., when a semaphore is fully acquired), so I don't see\nanother way of doing this. Contributions are very welcome if you do!\n\u003cbr\u003e\u003cbr\u003e\nThen if the queue needs to be created we call [`RPUSH`](https://redis.io/commands/rpush/) with the number of arguments\nequal to the `capacity` value used when initializing the semaphore instance. For a semaphore with\na capacity of 5, we call `RPUSH 1 1 1 1 1`, where the values are completely arbitrary.\n\nOnce the list/queue has been created, we [`BLPOP`](https://redis.io/commands/blpop/) to block until it's\n our turn. `BLPOP` is FIFO by default. We also make sure to specify the `max_sleep` based on the initialized\n semaphore instance setting. If nothing was passed we allow sleeping forever.\n\nOn `__aexit__` we run three commands in a pipelined query. We [`RPUSH`](https://redis.io/commands/rpush/) a `1`\nback into the queue to \"release\" the semaphore, and set an expiry on the queue and the string value we called\n`SETNX` on.\n\u003cbr\u003e\u003cbr\u003e\nThe expires are a half measure for dealing with dropped capacity. If a node holding the semaphore dies,\nthe capacity might never be returned. If, however, there is no one using the semaphore for the duration of the\nexpiry value, all values will be cleared, and the semaphore will be recreated at full capacity next time it's used.\nThe expiry is 30 seconds at the time of writing, but could be made configurable.\n\n### The token bucket implementation\n\nThe token bucket implementation is useful when you need to limit a process by\na time interval. For example, to 1 request per minute, or 50 requests every 10 seconds.\n\nThe implementation is forward-looking. It works out the time there *would have been*\ncapacity in the bucket for a given client and returns that time. From there we can\nasynchronously sleep until it's time to perform our rate limited action.\n\nThe flow can be broken down as follows:\n\n\u003cimg width=700 src=\"docs/token_bucket.png\"\u003e\u003c/img\u003e\n\nCall the [schedule Lua script](https://github.com/snok/self-limiters/blob/main/scripts/token_bucket.lua)\nwhich first [`GET`](https://redis.io/commands/get/)s the *state* of the bucket.\n\nThe bucket state contains the last time slot scheduled and the number of tokens left for that time slot.\nWith a capacity of 1, having a `tokens_left_for_slot` variable makes no sense, but if there's\ncapacity of 2 or more, it is possible that we will need to schedule multiple clients to the\nsame time slot.\n\nThe script then works out whether to decrement the `tokens_left_for_slot` value, or to\nincrement the time slot value wrt. the frequency variable.\n\nFinally, we store the bucket state again using [`SETEX`](https://redis.io/commands/setex/).\nThis allows us to store the state and set expiry at the same time. The default expiry\nis 30 at the time of writing, but could be made configurable.\n\nOne thing to note, is that this would not work if it wasn't for the fact that redis is single threaded,\nso Lua scripts on Redis are FIFO. Without this we would need locks and a lot more logic.\n\nThen we just sleep!\n\n# Contributing\n\nPlease do! Feedback on the implementation, issues, and PRs are all welcome. See [`CONTRIBUTING.md`](https://github.com/snok/self-limiters/blob/main/CONTRIBUTING.md) for more details.\n\nPlease also consider starring the repo to raise visibility.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnok%2Fself-limiters","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnok%2Fself-limiters","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnok%2Fself-limiters/lists"}