{"id":19665221,"url":"https://github.com/alttch/atasker","last_synced_at":"2025-04-28T22:31:04.085Z","repository":{"id":57412328,"uuid":"196765992","full_name":"alttch/atasker","owner":"alttch","description":"Python library for modern thread / multiprocessing pooling and task processing via asyncio","archived":false,"fork":false,"pushed_at":"2021-02-02T22:32:14.000Z","size":317,"stargazers_count":15,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-18T15:08:26.718Z","etag":null,"topics":["asyncio","multiprocessing","multithreading","priority-scheduling","thread","workers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alttch.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-07-13T21:00:11.000Z","updated_at":"2023-09-15T14:20:32.000Z","dependencies_parsed_at":"2022-08-27T23:50:55.773Z","dependency_job_id":null,"html_url":"https://github.com/alttch/atasker","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alttch%2Fatasker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alttch%2Fatasker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alttch%2Fatasker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alttch%2Fatasker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alttch","download_url":"https://codeload.github.com/alttch/atasker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251397576,"owners_count":21583034,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","multiprocessing","multithreading","priority-scheduling","thread","workers"],"created_at":"2024-11-11T16:21:39.474Z","updated_at":"2025-04-28T22:31:03.579Z","avatar_url":"https://github.com/alttch.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# atasker\nPython library for modern thread / multiprocessing pooling and task processing\nvia asyncio.\n\n\u003cimg src=\"https://img.shields.io/pypi/v/atasker.svg\" /\u003e\n\u003cimg src=\"https://img.shields.io/badge/license-Apache%202-blue.svg\" /\u003e\n\u003cimg src=\"https://img.shields.io/badge/python-3.5%20%7C%203.6%20%7C%203.7-blue.svg\" /\u003e\n\u003cimg src=\"https://img.shields.io/badge/-alpha-red.svg\" /\u003e\n\nWarning: **atasker** is not suitable for the lightweight tasks in high-load\nenvironments. For such projects it's highly recommended to use lightweight\nversion: [neotasker](https://github.com/alttch/neotasker)\n\nNo matter how your code is written, atasker automatically detects blocking\nfunctions and coroutines and launches them in a proper way, in a thread,\nasynchronous loop or in multiprocessing pool.\n\nTasks are grouped into pools. If there's no space in pool, task is being placed\ninto waiting queue according to their priority. Pool also has \"reserve\" for the\ntasks with priorities \"normal\" and higher. Tasks with \"critical\" priority are\nalways executed instantly.\n\nThis library is useful if you have a project with many similar tasks which\nproduce approximately equal CPU/memory load, e.g. API responses, scheduled\nresource state updates etc.\n\n## Install\n\n```bash\npip3 install atasker\n```\n\nSources: https://github.com/alttch/atasker\n\nDocumentation: https://atasker.readthedocs.io/\n\n## Why\n\n* asynchronous programming is a perfect way to make your code fast and reliable\n\n* multithreading programming is a perfect way to run blocking code in the\n  background\n\n**atasker** combines advantages of both ways: atasker tasks run in separate\nthreads however task supervisor and workers are completely asynchronous. But\nall their public methods are thread-safe.\n\n## Why not standard Python thread pool?\n\n* threads in a standard pool don't have priorities\n* workers\n\n## Why not standard asyncio loops?\n\n* compatibility with blocking functions\n* async workers\n\n## Why not concurrent.futures?\n\n**concurrent.futures** is a great standard Python library which allows you to\nexecute specified tasks in a pool of workers.\n\nFor thread-based tasks, **atasker** extends\n*concurrent.futures.ThreadPoolExecutor* functionality.\n\n**atasker** method *background_task* solves the same problem but in slightly\ndifferent way, adding priorities to the tasks, while *atasker* workers do\nabsolutely different job:\n\n* in *concurrent.futures* worker is a pool member which executes the single\n  specified task.\n\n* in *atasker* worker is an object, which continuously *generates* new tasks\n  with the specified interval or on external event, and executes them in thread\n  or multiprocessing pool.\n\n\n## Code examples\n\n### Start/stop\n\n```python\n\nfrom atasker import task_supervisor\n\n# set pool size\ntask_supervisor.set_thread_pool(pool_size=20, reserve_normal=5, reserve_high=5)\ntask_supervisor.start()\n# ...\n# start workers, other threads etc.\n# ...\n# optionally block current thread\ntask_supervisor.block()\n\n# stop from any thread\ntask_supervisor.stop()\n```\n\n### Background task\n\n```python\nfrom atasker import background_task, TASK_LOW, TASK_HIGH, wait_completed\n\n# with annotation\n@background_task\ndef mytask():\n    print('I am working in the background!')\n    return 777\n\ntask = mytask()\n\n# optional\nresult = wait_completed(task)\n\nprint(task.result) # 777\nprint(result) # 777\n\n# with manual decoration\ndef mytask2():\n    print('I am working in the background too!')\n\ntask = background_task(mytask2, priority=TASK_HIGH)()\n```\n### Async tasks\n\n```python\n# new asyncio loop is automatically created in own thread\na1 = task_supervisor.create_aloop('myaloop', default=True)\n\nasync def calc(a):\n    print(a)\n    await asyncio.sleep(1)\n    print(a * 2)\n    return a * 3\n\n# call from sync code\n\n# put coroutine\ntask = background_task(calc)(1)\n\nwait_completed(task)\n\n# run coroutine and wait for result\nresult = a1.run(calc(1))\n```\n\n### Worker examples\n\n```python\nfrom atasker import background_worker, TASK_HIGH\n\n@background_worker\ndef worker1(**kwargs):\n    print('I am a simple background worker')\n\n@background_worker\nasync def worker_async(**kwargs):\n    print('I am async background worker')\n\n@background_worker(interval=1)\ndef worker2(**kwargs):\n    print('I run every second!')\n\n@background_worker(queue=True)\ndef worker3(task, **kwargs):\n    print('I run when there is a task in my queue')\n\n@background_worker(event=True, priority=TASK_HIGH)\ndef worker4(**kwargs):\n    print('I run when triggered with high priority')\n\nworker1.start()\nworker_async.start()\nworker2.start()\nworker3.start()\nworker4.start()\n\nworker3.put_threadsafe('todo1')\nworker4.trigger_threadsafe()\n\nfrom atasker import BackgroundIntervalWorker\n\nclass MyWorker(BackgroundIntervalWorker):\n\n    def run(self, **kwargs):\n        print('I am custom worker class')\n\nworker5 = MyWorker(interval=0.1, name='worker5')\nworker5.start()\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falttch%2Fatasker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falttch%2Fatasker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falttch%2Fatasker/lists"}