{"id":15658460,"url":"https://github.com/long2ice/rearq","last_synced_at":"2025-04-12T19:46:10.985Z","repository":{"id":44963402,"uuid":"267289972","full_name":"long2ice/rearq","owner":"long2ice","description":"A distributed task queue built with asyncio and redis, with built-in web interface","archived":false,"fork":false,"pushed_at":"2025-03-11T14:29:16.000Z","size":3097,"stargazers_count":153,"open_issues_count":3,"forks_count":10,"subscribers_count":5,"default_branch":"dev","last_synced_at":"2025-04-04T00:09:28.076Z","etag":null,"topics":["arq","asyncio","distributed","mysql","postgresql","queue","redis","rq","task"],"latest_commit_sha":null,"homepage":"https://rearq.long2ice.io/rearq","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/long2ice.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"custom":["https://sponsor.long2ice.io"]}},"created_at":"2020-05-27T10:31:50.000Z","updated_at":"2025-03-14T12:48:43.000Z","dependencies_parsed_at":"2024-10-28T19:42:46.707Z","dependency_job_id":"76d24901-f68c-420c-806a-bf2f01c7d35b","html_url":"https://github.com/long2ice/rearq","commit_stats":{"total_commits":205,"total_committers":3,"mean_commits":68.33333333333333,"dds":"0.15121951219512197","last_synced_commit":"cdb2fe2e9f6f0c0c9fda0eafa0d90a24da7a70cd"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/long2ice%2Frearq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/long2ice%2Frearq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/long2ice%2Frearq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/long2ice%2Frearq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/long2ice","download_url":"https://codeload.github.com/long2ice/rearq/tar.gz/refs/heads/dev","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248625491,"owners_count":21135513,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arq","asyncio","distributed","mysql","postgresql","queue","redis","rq","task"],"created_at":"2024-10-03T13:12:31.294Z","updated_at":"2025-04-12T19:46:10.959Z","avatar_url":"https://github.com/long2ice.png","language":"Python","funding_links":["https://sponsor.long2ice.io"],"categories":[],"sub_categories":[],"readme":"# ReArq\n\n[![image](https://img.shields.io/pypi/v/rearq.svg?style=flat)](https://pypi.python.org/pypi/rearq)\n[![image](https://img.shields.io/github/license/long2ice/rearq)](https://github.com/long2ice/rearq)\n[![image](https://github.com/long2ice/rearq/workflows/pypi/badge.svg)](https://github.com/long2ice/rearq/actions?query=workflow:pypi)\n[![image](https://github.com/long2ice/rearq/workflows/ci/badge.svg)](https://github.com/long2ice/rearq/actions?query=workflow:ci)\n\n## Introduction\n\nReArq is a distributed task queue with asyncio and redis, which rewrite from [arq](https://github.com/samuelcolvin/arq)\nto make improvement and include web interface.\n\nYou can try [Demo Online](https://rearq.long2ice.io) here.\n\n## Features\n\n- AsyncIO support, easy integration with [FastAPI](https://github.com/tiangolo/fastapi).\n- Delay task, cron task and async task support.\n- Full-featured build-in web interface.\n- Built-in distributed task lock to make same task only run one at the same time.\n- Other powerful features to be discovered.\n\n## Screenshots\n\n![dashboard](./images/dashboard.png)\n![worker](./images/worker.png)\n![task](./images/task.png)\n![job](./images/job.png)\n![result](./images/result.png)\n\n## Requirements\n\n- Redis \u003e= 5.0\n\n## Install\n\nUse MySQL backend:\n\n```shell\npip install rearq[mysql]\n```\n\nUse PostgreSQL backend:\n\n```shell\npip install rearq[postgres]\n```\n\n## Quick Start\n\n### Task Definition\n\n```python\n# main.py\nfrom rearq import ReArq\n\nrearq = ReArq(db_url='mysql://root:123456@127.0.0.1:3306/rearq')\n\n\n@rearq.on_shutdown\nasync def on_shutdown():\n    # you can do some clean work here like close db and so on...\n    print(\"shutdown\")\n\n\n@rearq.on_startup\nasync def on_startup():\n    # you should do some initialization work here\n    print(\"startup\")\n    # you must init Tortoise ORM here\n    await Tortoise.init(\n        db_url=settings.DB_URL,\n        modules={\"rearq\": [\"rearq.server.models\"]},\n    )\n\n\n@rearq.task(queue=\"q1\")\nasync def add(self, a, b):\n    return a + b\n\n\n@rearq.task(cron=\"*/5 * * * * * *\")  # run task per 5 seconds\nasync def timer(self):\n    return \"timer\"\n```\n\n### Run rearq worker\n\n```shell\n\u003e rearq main:rearq worker -q q1 -q q2 # consume tasks from q1 and q2 as the same time\n```\n\n```log\n2021-03-29 09:54:50.464 | INFO     | rearq.worker:_main:95 - Start worker success with queue: rearq:queue:default\n2021-03-29 09:54:50.465 | INFO     | rearq.worker:_main:96 - Registered tasks: add, sleep, timer_add\n2021-03-29 09:54:50.465 | INFO     | rearq.worker:log_redis_info:86 - redis_version=6.2.1 mem_usage=1.43M clients_connected=5 db_keys=6\n```\n\n### Run rearq timer\n\nIf you have timing task or delay task, you should run another command also:\n\n```shell\n\u003e rearq main:rearq timer\n```\n\n```log\n2021-03-29 09:54:43.878 | INFO     | rearq.worker:_main:275 - Start timer success\n2021-03-29 09:54:43.887 | INFO     | rearq.worker:_main:277 - Registered timer tasks: timer_add\n2021-03-29 09:54:43.894 | INFO     | rearq.worker:log_redis_info:86 - redis_version=6.2.1 mem_usage=1.25M clients_connected=2 db_keys=6\n```\n\nAlso, you can run timer with worker together by `rearq main:rearq worker -t`.\n\n### Integration in FastAPI\n\n```python\nfrom fastapi import FastAPI\n\napp = FastAPI()\n\n\n@app.on_event(\"startup\")\nasync def startup() -\u003e None:\n    await Tortoise.init(\n        db_url=settings.DB_URL,\n        modules={\"rearq\": [\"rearq.server.models\"]},\n    )\n\n\n@app.on_event(\"shutdown\")\nasync def shutdown() -\u003e None:\n    await rearq.close()\n\n\n# then run task in view\n@app.get(\"/test\")\nasync def test():\n    job = await add.delay(args=(1, 2))\n    # or\n    job = await add.delay(kwargs={\"a\": 1, \"b\": 2})\n    # or\n    job = await add.delay(1, 2)\n    # or\n    job = await add.delay(a=1, b=2)\n    result = await job.result(timeout=5)  # wait result for 5 seconds\n    print(result.result)\n    return result\n```\n\n## Start web interface\n\n```shell\n\u003e rearq main:rearq server\nUsage: rearq server [OPTIONS]\n\n  Start rest api server.\n\nOptions:\n  --host TEXT         Listen host.  [default: 0.0.0.0]\n  -p, --port INTEGER  Listen port.  [default: 8000]\n  -h, --help          Show this message and exit..\n```\n\nAfter server run, you can visit [https://127.0.0.1:8000/docs](https://127.0.0.1:8000/docs) to see all apis\nand [https://127.0.0.1:8000](https://127.0.0.1:8000) to see web interface.\n\nOther options will pass into `uvicorn` directly, such as `--root-path` etc.\n\n```shell\nrearq main:rearq server --host 0.0.0.0 --root-path /rearq\n```\n\n### Mount as FastAPI sub app\n\nYou can also mount rearq server as FastAPI sub app.\n\n```python\n\nfrom fastapi import FastAPI\n\nfrom examples.tasks import rearq\nfrom rearq.server.app import app as rearq_app\n\napp = FastAPI()\n\napp.mount(\"/rearq\", rearq_app)\nrearq_app.set_rearq(rearq)\n```\n\n### Start worker inside app\n\nYou can also start worker inside your app.\n\n```python\n@app.on_event(\"startup\")\nasync def startup():\n    await rearq.init()\n    await rearq_app.start_worker(with_timer=True, block=False)\n```\n\n## ThanksTo\n\n- [arq](https://github.com/samuelcolvin/arq), Fast job queuing and RPC in python with asyncio and redis.\n\n## License\n\nThis project is licensed under the [Apache-2.0](./LICENSE) License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flong2ice%2Frearq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flong2ice%2Frearq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flong2ice%2Frearq/lists"}