{"id":28407063,"url":"https://github.com/sirkonst/async-reduce","last_synced_at":"2025-06-29T13:32:19.025Z","repository":{"id":53792484,"uuid":"179439119","full_name":"sirkonst/async-reduce","owner":"sirkonst","description":"Reducer for similar simultaneously coroutines","archived":false,"fork":false,"pushed_at":"2025-01-18T18:28:08.000Z","size":58,"stargazers_count":21,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-06-02T09:11:28.948Z","etag":null,"topics":["async-await","asynchronous","asyncio","python","python3","reduce","reducer"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sirkonst.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-04-04T06:53:01.000Z","updated_at":"2025-01-18T18:28:10.000Z","dependencies_parsed_at":"2025-01-18T19:24:52.625Z","dependency_job_id":"f3e7f0bf-0d9d-4e93-b98b-ced30a8af697","html_url":"https://github.com/sirkonst/async-reduce","commit_stats":{"total_commits":23,"total_committers":2,"mean_commits":11.5,"dds":0.04347826086956519,"last_synced_commit":"bbc049e1b08242d54957e54209aa86e6b793e3d6"},"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/sirkonst/async-reduce","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sirkonst%2Fasync-reduce","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sirkonst%2Fasync-reduce/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sirkonst%2Fasync-reduce/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sirkonst%2Fasync-reduce/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sirkonst","download_url":"https://codeload.github.com/sirkonst/async-reduce/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sirkonst%2Fasync-reduce/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262600536,"owners_count":23335087,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async-await","asynchronous","asyncio","python","python3","reduce","reducer"],"created_at":"2025-06-02T00:08:59.355Z","updated_at":"2025-06-29T13:32:19.010Z","avatar_url":"https://github.com/sirkonst.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Python versions](https://img.shields.io/badge/python-3.7%2C%203.8%2C%203.9%2C%203.10%2C%203.11%2C%203.12%2C%203.13-green.svg)]()\n[![PyPI version](https://badge.fury.io/py/async-reduce.svg)](https://pypi.org/project/async-reduce/)\n\n\nAbout Async-Reduce\n==================\n\n``async_reduce(coroutine)`` allows aggregate all *similar simultaneous*\nready to run `coroutine`s and reduce to running **only one** `coroutine`.\nOther aggregated `coroutine`s will get result from single `coroutine`.\n\nIt can boost application performance in highly competitive execution of the\nsimilar asynchronous operations and reduce load for inner systems.\n\n\nQuick example\n-------------\n\n```python\nfrom async_reduce import async_reduce\n\n\nasync def fetch_user_data(user_id: int) -\u003e dict:\n    \"\"\"\" Get user data from inner service \"\"\"\n    url = 'http://inner-service/user/{}'.format(user_id)\n\n    return await http.get(url, timeout=10).json()\n\n\n@web_server.router('/users/(\\d+)')\nasync def handler_user_detail(request, user_id: int):\n    \"\"\" Handler for get detail information about user \"\"\"\n\n    # all simultaneous requests of fetching user data for `user_id` will\n    # reduced to single request\n    user_data = await async_reduce(\n        fetch_user_data(user_id)\n    )\n\n    # sometimes ``async_reduce`` cannot detect similar coroutines and\n    # you should provide special argument `ident` for manually determination\n    user_statistics = await async_reduce(\n        DataBase.query('user_statistics').where(id=user_id).fetch_one(),\n        ident='db_user_statistics:{}'.format(user_id)\n    )\n\n    return Response(...)\n```\n\nIn that example without using ``async_reduce`` if client performs **N**\nsimultaneous requests like `GET http://web_server/users/42` *web_server*\nperforms **N** requests to *inner-service* and **N** queries to *database*.\nIn total: **N** simultaneous requests emits **2 * N** requests to inner systems.\n\nWith ``async_reduce`` if client performs **N** simultaneous requests *web_server*\nperforms **one** request to *inner-service* and **one** query to *database*.\nIn total: **N** simultaneous requests emit only **2** requests to inner systems.\n\nSee other real [examples](https://github.com/sirkonst/async-reduce/tree/master/examples).\n\n\nSimilar coroutines determination\n--------------------------------\n\n``async_reduce(coroutine)`` tries to detect similar coroutines by hashing\nlocal variables bounded on call. It does not work correctly if:\n\n* one of the arguments is not hashable\n* coroutine function is a method of class with specific state (like ORM)\n* coroutine function has closure to unhashable variable\n\nYou can disable auto-determination by setting custom key to argument ``ident``.\n\n\nUse as decorator\n----------------\n\nAlso library provide special decorator ``@async_reduceable()``, example:\n\n```python\nfrom async_reduce import async_reduceable\n\n\n@async_reduceable()\nasync def fetch_user_data(user_id: int) -\u003e dict:\n    \"\"\"\" Get user data from inner service \"\"\"\n    url = 'http://inner-servicce/user/{}'.format(user_id)\n\n    return await http.get(url, timeout=10).json()\n\n\n@web_server.router('/users/(\\d+)')\nasync def handler_user_detail(request, user_id: int):\n    \"\"\" Handler for get detail information about user \"\"\"\n    return await fetch_user_data(user_id)\n```\n\n\nHooks\n-----\n\nLibrary supports hooks. Add-on hooks:\n\n* **DebugHooks** - print about all triggered hooks\n* **StatisticsOverallHooks** - general statistics on the use of `async_reduce`\n* **StatisticsDetailHooks** - like `StatisticsOverallHooks` but detail statistics\nabout all `coroutine` processed by `async_reduce`\n\nExample:\n\n```python\nfrom async_reduce import AsyncReducer\nfrom async_reduce.hooks import DebugHooks\n\n# define custom async_reduce with hooks\nasync_reduce = AsyncReducer(hooks=DebugHooks())\n\n\nasync def handler_user_detail(request, user_id: int):\n    user_data = await async_reduce(fetch_user_data(user_id))\n```\n\nSee more detail example in [examples/example_hooks.py](https://github.com/sirkonst/async-reduce/blob/master/examples/example_hooks.py).\n\nYou can write custom hooks via inherit from [BaseHooks](https://github.com/sirkonst/async-reduce/blob/master/async_reduce/hooks/base.py).\n\n\nCaveats\n-------\n\n* If single `coroutine` raises exceptions all aggregated `coroutine`s will get\nsame exception too\n\n* If single `coroutine` is stuck all aggregated `coroutine`s will stuck too.\nLimit execution time for `coroutine` and add retries (optional) to avoid it.\n\n* Be careful when return mutable value from `coroutine` because single value\nwill shared. Prefer to use non-mutable value as coroutine return.\n\n\nDevelopment\n-----------\n\nSee [DEVELOPMENT.md](https://github.com/sirkonst/async-reduce/blob/master/DEVELOPMENT.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsirkonst%2Fasync-reduce","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsirkonst%2Fasync-reduce","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsirkonst%2Fasync-reduce/lists"}