{"id":13721231,"url":"https://github.com/achimnol/aiotools","last_synced_at":"2025-10-19T01:46:29.437Z","repository":{"id":19905130,"uuid":"88231996","full_name":"achimnol/aiotools","owner":"achimnol","description":"Idiomatic asyncio utilties","archived":false,"fork":false,"pushed_at":"2025-04-07T16:46:03.000Z","size":503,"stargazers_count":158,"open_issues_count":12,"forks_count":13,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-05-12T01:45:25.900Z","etag":null,"topics":["asyncio","library","python","utils"],"latest_commit_sha":null,"homepage":"https://aiotools.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/achimnol.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-04-14T04:10:54.000Z","updated_at":"2025-03-28T01:37:36.000Z","dependencies_parsed_at":"2022-08-07T09:15:43.908Z","dependency_job_id":"de78c508-d289-468a-8211-704c69a4d860","html_url":"https://github.com/achimnol/aiotools","commit_stats":{"total_commits":298,"total_committers":8,"mean_commits":37.25,"dds":0.03355704697986572,"last_synced_commit":"6d8a93ed0418d61e7c40c7f5c06b061af4581c06"},"previous_names":[],"tags_count":63,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/achimnol%2Faiotools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/achimnol%2Faiotools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/achimnol%2Faiotools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/achimnol%2Faiotools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/achimnol","download_url":"https://codeload.github.com/achimnol/aiotools/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254364266,"owners_count":22058877,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","library","python","utils"],"created_at":"2024-08-03T01:01:14.250Z","updated_at":"2025-10-19T01:46:29.431Z","avatar_url":"https://github.com/achimnol.png","language":"Python","funding_links":[],"categories":["其他"],"sub_categories":[],"readme":"aiotools\n========\n\n[![PyPI release version](https://badge.fury.io/py/aiotools.svg)](https://pypi.org/project/aiotools/)\n![Supported Python versions](https://img.shields.io/pypi/pyversions/aiotools.svg)\n[![CI Status](https://github.com/achimnol/aiotools/actions/workflows/ci.yml/badge.svg)](https://github.com/achimnol/aiotools/actions/workflows/ci.yml)\n[![Code Coverage](https://codecov.io/gh/achimnol/aiotools/branch/master/graph/badge.svg)](https://codecov.io/gh/achimnol/aiotools)\n\nIdiomatic asyncio utilities\n\n\nModules\n-------\n\n* [Safe Cancellation](https://aiotools.readthedocs.io/en/latest/aiotools.cancel.html)\n* [Async Context Manager](https://aiotools.readthedocs.io/en/latest/aiotools.context.html)\n* [Async Defer](https://aiotools.readthedocs.io/en/latest/aiotools.defer.html)\n* [Async Fork](https://aiotools.readthedocs.io/en/latest/aiotools.fork.html)\n* [Async Functools](https://aiotools.readthedocs.io/en/latest/aiotools.func.html)\n* [Async Itertools](https://aiotools.readthedocs.io/en/latest/aiotools.iter.html)\n* [Async Server](https://aiotools.readthedocs.io/en/latest/aiotools.server.html)\n* [Async Timer](https://aiotools.readthedocs.io/en/latest/aiotools.timer.html)\n* [TaskContext](https://aiotools.readthedocs.io/en/latest/aiotools.taskcontext.html)\n* [TaskScope](https://aiotools.readthedocs.io/en/latest/aiotools.taskscope.html)\n* (alias of TaskScope) [Supervisor](https://aiotools.readthedocs.io/en/latest/aiotools.supervisor.html)\n* (deprecated since 2.0) [(Persistent)TaskGroup](https://aiotools.readthedocs.io/en/latest/aiotools.taskgroup.html)\n* [High-level Coroutine Utilities](https://aiotools.readthedocs.io/en/latest/aiotools.utils.html)\n\nFull API documentation: https://aiotools.readthedocs.io\n\n\nSee also\n--------\n\n* [anyio](https://github.com/agronholm/anyio): High level asynchronous concurrency and networking framework that works on top of either Trio or asyncio\n* [trio](https://github.com/python-trio/trio): An alternative implementation of asyncio focusing on structured concurrency\n* [aiometer](https://github.com/florimondmanca/aiometer): A Python concurrency scheduling library, compatible with asyncio and trio.\n* [aiojobs](https://github.com/aio-libs/aiojobs): A concurrency-limiting, task-shielding scheduler for asyncio tasks for graceful shutdown\n\nCurrently aiotools targets the vanilly asyncio ecosystem only\n(with some tight coupling with asyncio internals),\nbut you may find similar problem definitions and alternative solutions in the above libraries.\n\n\nExamples\n--------\n\nBelow are some highlighted usecases of aiotools.\nPlease refer more detailed logic and backgrounds in [the documentation](https://aiotools.readthedocs.io).\n\n### Safe Cancellation\n\nConsider the following commonly used pattern:\n```python\ntask = asyncio.create_task(...)\ntask.cancel()\nawait task  # PROBLEM: would it raise CancelledError or not? should we propagate it or not?\n```\n\nIt has been the reponsibility of the author of tasks and the caller of them to\ncoordinate whether to re-raise injected cancellation error.\n\nNow we can use the structured cancellation introduced in Python 3.11:\n```python\ntask = asyncio.create_task(...)\nawait aiotools.cancel_and_wait(task)\n```\nwhich will re-raise the cancellation when there is an external cancellation\nrequest and absorb it otherwise.\n\nRelying on this API whenever you need to cancel asyncio tasks will make your\ncodebase more consistent because you no longer need to decide whether to\n(re-)raise or suppress `CancelledError` in your task codes.\n\nYou may also combine `cancel_and_wait()` with `ShieldScope` to guard\na block of codes from cancellation in the middle but defer the cancellation\nto the end of the block.\n\n```python\nasync def work():\n    try:\n        ...\n    except asyncio.CancelledError:\n        with aiotools.ShieldScope():\n            await cleanup()  # any async code here is not affected by multiple cancellation\n            raise\n\nasync def parent():\n    work_task = asyncio.create_task(work())\n    ...\n    await cancel_and_wait(work_task)\n\nparent_task = asyncio.create_task(parent())\n...\nawait cancel_and_wait(parent_task)  # it may trigger double cancellation, but it will return after the shielded block completes.\n```\n\n### Async Context Manager\n\nThis is an asynchronous version of `contextlib.contextmanager` to make it\neasier to write asynchronous context managers without creating boilerplate\nclasses.\n\n```python\nimport asyncio\nimport aiotools\n\n@aiotools.actxmgr\nasync def mygen(a):\n    await asyncio.sleep(1)\n    yield a + 1\n    await asyncio.sleep(1)\n\nasync def somewhere():\n    async with mygen(1) as b:\n        assert b == 2\n```\n\nNote that you need to wrap `yield` with a try-finally block to\nensure resource releases (e.g., locks), even in the case when\nan exception is ocurred inside the async-with block.\n\n```python\nimport asyncio\nimport aiotools\n\nlock = asyncio.Lock()\n\n@aiotools.actxmgr\nasync def mygen(a):\n    await lock.acquire()\n    try:\n        yield a + 1\n    finally:\n        lock.release()\n\nasync def somewhere():\n    try:\n        async with mygen(1) as b:\n            raise RuntimeError('oops')\n    except RuntimeError:\n        print('caught!')  # you can catch exceptions here.\n```\n\nYou can also create a group of async context managers, which\nare entered/exited all at once using `asyncio.gather()`.\n\n```python\nimport asyncio\nimport aiotools\n\n@aiotools.actxmgr\nasync def mygen(a):\n    yield a + 10\n\nasync def somewhere():\n    ctxgrp = aiotools.actxgroup(mygen(i) for i in range(10))\n    async with ctxgrp as values:\n        assert len(values) == 10\n        for i in range(10):\n            assert values[i] == i + 10\n```\n\n\n### Async Server\n\nThis implements a common pattern to launch asyncio-based server daemons.\n\n```python\nimport asyncio\nimport aiotools\n\nasync def echo(reader, writer):\n    data = await reader.read(100)\n    writer.write(data)\n    await writer.drain()\n    writer.close()\n\n@aiotools.server\nasync def myworker(loop, pidx, args):\n    server = await asyncio.start_server(echo, '0.0.0.0', 8888, reuse_port=True)\n    print(f'[{pidx}] started')\n    yield  # wait until terminated\n    server.close()\n    await server.wait_closed()\n    print(f'[{pidx}] terminated')\n\nif __name__ == '__main__':\n    # Run the above server using 4 worker processes.\n    aiotools.start_server(myworker, num_workers=4)\n```\n\nIt handles SIGINT/SIGTERM signals automatically to stop the server,\nas well as lifecycle management of event loops running on multiple processes.\nInternally it uses `aiotools.fork` module to get kernel support to resolve\npotential signal/PID related races via PID file descriptors on supported versions\n(Python 3.9+ and Linux kernel 5.4+).\n\n\n### Async TaskScope\n\nTaskScope is a variant of TaskGroup which ensures all child tasks run to completion\n(either having results or exceptions) unless the context manager is cancelled.\n\nThis could be considered as a safer version (in terms of lifecycle tracking) of\n`asyncio.gather(*, return_exceptions=True)` and a more convenient version of it\nbecause you can decouple the timing of task creation and their scheduling\nwithin the TaskScope context.\n\n```python\nimport aiotools\n\nasync def do():\n    async with aiotools.TaskScope() as ts:\n        ts.create_task(...)\n        ts.create_task(...)\n        # each task will run to completion regardless sibling failures.\n        ...\n    # at this point, all subtasks are either cancelled or done.\n```\n\nYou may customize exception handler for each scope to receive and process\nunhandled exceptions in child tasks.  For use in long-running server contexts,\nTaskScope does not store any exceptions or results by itself.\n\nSee also high-level coroutine utilities such as `as_completed_safe()`,\n`gather_safe()`, and `race()` functions in the utils module.\n\nTaskScope itself and these utilities integrate with\n[the call-graph inspection](https://docs.python.org/3/library/asyncio-graph.html)\nintroduced in Python 3.14.\n\n\n### Async Timer\n\n```python\nimport aiotools\n\ni = 0\n\nasync def mytick(interval):\n    print(i)\n    i += 1\n\nasync def somewhere():\n    task = aiotools.create_timer(mytick, 1.0)\n    ...\n    await aiotools.cancel_and_wait(task)\n```\n\nThe returned `task` is an `asyncio.Task` object.\nTo stop the timer, it should be cancelled explicitly.\nUse `cancel_and_wait()` to ensure complete shutdown of any ongoing tick tasks.\nTo make your timer function to be cancellable, add a try-except clause\ncatching `asyncio.CancelledError` since we use it as a termination\nsignal.\n\nYou may add `TimerDelayPolicy` argument to control the behavior when the\ntimer-fired task takes longer than the timer interval.\n`DEFAULT` is to accumulate them and cancel all the remainings at once when\nthe timer is cancelled.\n`CANCEL` is to cancel any pending previously fired tasks on every interval.\n\n```python\nimport asyncio\nimport aiotools\n\nasync def mytick(interval):\n    await asyncio.sleep(100)  # cancelled on every next interval.\n\nasync def somewhere():\n    t = aiotools.create_timer(mytick, 1.0, aiotools.TimerDelayPolicy.CANCEL)\n    ...\n    await aiotools.cancel_and_wait(t)\n```\n\n#### Virtual Clock\n\nIt provides a virtual clock that advances the event loop time instantly upon\nany combination of `asyncio.sleep()` calls in multiple coroutine tasks,\nby temporarily patching the event loop selector.\n\nThis is also used in [our timer test suite](https://github.com/achimnol/aiotools/blob/master/tests/test_timer.py).\n\n```python\nimport aiotools\nimport pytest\n\n@pytest.mark.asyncio\nasync def test_sleeps():\n    loop = aiotools.compat.get_running_loop()\n    vclock = aiotools.VirtualClock()\n    with vclock.patch_loop():\n        print(loop.time())  # -\u003e prints 0\n        await asyncio.sleep(3600)\n        print(loop.time())  # -\u003e prints 3600\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fachimnol%2Faiotools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fachimnol%2Faiotools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fachimnol%2Faiotools/lists"}