{"id":20030617,"url":"https://github.com/peterding/chihuo","last_synced_at":"2025-07-23T10:05:26.857Z","repository":{"id":56953850,"uuid":"385543314","full_name":"PeterDing/chihuo","owner":"PeterDing","description":"An Asynchronous Task Running Engine","archived":false,"fork":false,"pushed_at":"2023-02-16T10:15:43.000Z","size":98,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-08T18:17:30.674Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PeterDing.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-13T09:07:46.000Z","updated_at":"2021-10-22T08:13:11.000Z","dependencies_parsed_at":"2024-11-13T09:38:42.517Z","dependency_job_id":null,"html_url":"https://github.com/PeterDing/chihuo","commit_stats":{"total_commits":37,"total_committers":1,"mean_commits":37.0,"dds":0.0,"last_synced_commit":"1e8c2cb5e0e5be7c8caa45b7f4a61afd131d3a2e"},"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"purl":"pkg:github/PeterDing/chihuo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PeterDing%2Fchihuo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PeterDing%2Fchihuo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PeterDing%2Fchihuo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PeterDing%2Fchihuo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PeterDing","download_url":"https://codeload.github.com/PeterDing/chihuo/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PeterDing%2Fchihuo/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266657954,"owners_count":23963602,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-23T02:00:09.312Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T09:27:22.271Z","updated_at":"2025-07-23T10:05:26.837Z","avatar_url":"https://github.com/PeterDing.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# chihuo: An Asynchronous Task Running Engine\n\nchihuo is an asynchronous task running engine. It runs asynchronous tasks concurrently\nfrom a queue which can be at memory, Redis, Lodis or Some MQ.\n\n## The Loop\n\n`ChihuoLoop` is the asynchronous engine core which responses to connect the backend queue\nand distributes every tasks to the global event loop.\n\nFor a process, the chihuo create only one global event loop to run all asynchronous\ntasks and the loop is running forever.\n\n- `ChihuoLoop.start` method\n\n  When a `ChihuoLoop` starts, it needs to add some primary tasks to the backend queue.\n  The `start` function is the place to fire these priorities.\n\n- `ChihuoLoop.add_task` method\n\n  At anywhere and anytime, we can use the `add_task` method to add a task to backend queue.\n  The task discription must be the tuple as `(task_id: str, task_info: jsonifiable object_)`\n  e.g. `add_task((\"task1\", [1, 1+10]), (\"task2\", [2, 2+10]))`\n\n- `ChihuoLoop.make_task` method\n\n  Each classes which have implemented the `ChihuoLoop` class must to implement the `make_task`\n  method for receiving the task's information that is added by `ChihuoLoop.add_task` to\n  the backend queue.\n\n- `ChihuoLoop.task_finish` method\n\n  When a task is finished and is unneeded at future, we can tag the task as the `finished` state\n  using the `task_finish` method. e.g. `task_finish(task_id)`. The task that is at `finished`\n  state can't be add to backend queue again.\n\n- `ChihuoLoop.task_unfinish` method\n\n  When a task is at the `finished` state and we need to add it to queue again, we can use\n  the `task_unfinish` method to set the task state to `unfinish`. Then, we can use\n  `ChihuoLoop.add_task` to add the task to queue again.\n\n- `ChihuoLoop.stop` property\n\n  When the property `stop` to be set as True, the `ChihuoLoop` will stop pull the tasks from\n  queue and exit.\n\n## Demo\n\n```python\n# filename: abc.py\n\nfrom chihuo import ChihuoLoop\nfrom chihuo.common import SERVER_DEFAULT_CONFIG_PATH\nimport chihuo\n\n\nclass MyTasksA(ChihuoLoop):\n\n    NAME = \"my-tasksA\"  # The unique id for backend queue server\n    CONCURRENCY = 10   # The number of tasks running concurrently\n    SERVER_CONFIG = SERVER_DEFAULT_CONFIG_PATH\n\n    def __init__(self):\n        super().__init__()\n\n    async def start(self):\n        for id in range(10):\n            await self.add_task((str(i), {i ** 2}))\n\n    async def make_task(self, task_id, task):\n        print(f\"my-task-A: {task_id}\")\n        print(f\"power of {i} is {task}\")\n\n        # Talk chihuo that the task is finished\n        await self.task_finish(i)\n\n\nclass MyTasksB(ChihuoLoop):\n\n    NAME = \"my-tasksB\"  # The unique id for the loop\n    CONCURRENCY = 10   # The number of tasks running concurrently\n    SERVER_CONFIG = SERVER_DEFAULT_CONFIG_PATH\n\n    def __init__(self):\n        super().__init__()\n\n    async def start(self):\n        for id in range(10):\n            await self.add_task((str(i), {i ** 2}))\n\n    async def make_task(self, task_id, task):\n        print(f\"my-task-B: {task_id}\")\n        print(f\"power of {i} is {task}\")\n\n        # Talk chihuo that the task is finished\n        await self.task_finish(i)\n\n\nif __name__ == '__main__':\n    chihuo.run(MyTasksA, MyTasksB)\n```\n\nTo Run the project, firstly we need to launch a backend queue server.\nHere, we use a Lodis instance as default server.\n\nStart a Lodis server:\n\n```\nLODIS_DB_PATH=/data/test-lodis LODIS_IP_PORT=\"127.0.0.1:8311\" nohup /path/to/lodis \u0026\n```\n\nThen, we write a configure file as `./server.json` (the `SERVER_DEFAULT_CONFIG_PATH`)\nto connect the backend for our `MyTasksA` and `MyTasksB`.\n\n```\nbackend=lodis\nip=127.0.0.1\nport=8311\n```\n\nNow, we can run the script.\n\n```\npython3 abc.py\n```\n\n## Shutdown the Tasks\n\nDefaultly, the global loop is running forever and we need to shutdown the loop by sending\na `INT (interrupt)` signal to the process. After the process received The `INT` signal, it will\nset the property `stop` of instance which is implemented from `ChihuoLoop` to True and the\ninstance will stop all running tasks and send them back to queue, finally exit.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeterding%2Fchihuo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpeterding%2Fchihuo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeterding%2Fchihuo/lists"}