{"id":48330897,"url":"https://github.com/attakay78/fastapi-taskflow","last_synced_at":"2026-04-25T00:02:10.132Z","repository":{"id":349244810,"uuid":"1201502905","full_name":"Attakay78/fastapi-taskflow","owner":"Attakay78","description":"Turn FastAPI BackgroundTasks into a production-ready task system. Retries, Resiliency, control, and visibility without workers or brokers.","archived":false,"fork":false,"pushed_at":"2026-04-14T14:59:41.000Z","size":3049,"stargazers_count":22,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-14T16:31:34.677Z","etag":null,"topics":["background-tasks","fastapi","observability","resiliency","retries","task-management","task-manager","tasks","tasks-scheduler","visibility"],"latest_commit_sha":null,"homepage":"https://attakay78.github.io/fastapi-taskflow/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Attakay78.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-04T19:07:03.000Z","updated_at":"2026-04-14T14:58:26.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Attakay78/fastapi-taskflow","commit_stats":null,"previous_names":["attakay78/fastapi-taskflow"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/Attakay78/fastapi-taskflow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Attakay78%2Ffastapi-taskflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Attakay78%2Ffastapi-taskflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Attakay78%2Ffastapi-taskflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Attakay78%2Ffastapi-taskflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Attakay78","download_url":"https://codeload.github.com/Attakay78/fastapi-taskflow/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Attakay78%2Ffastapi-taskflow/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32245152,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T13:21:15.438Z","status":"ssl_error","status_checked_at":"2026-04-24T13:21:15.005Z","response_time":64,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["background-tasks","fastapi","observability","resiliency","retries","task-management","task-manager","tasks","tasks-scheduler","visibility"],"created_at":"2026-04-05T01:02:44.454Z","updated_at":"2026-04-25T00:02:10.125Z","avatar_url":"https://github.com/Attakay78.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/Attakay78/fastapi-taskflow/main/docs/banner.svg\" alt=\"fastapi-taskflow\" width=\"380\" /\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003cstrong\u003eTurn FastAPI BackgroundTasks into a production-ready task system.\u003c/strong\u003e\u003cbr/\u003e\n  Retries, control, resiliency and visibility without workers or brokers.\n\u003c/p\u003e\n\n---\n\nFastAPI's `BackgroundTasks` handles simple fire-and-forget work well. But in real applications you quickly hit the same gaps: tasks fail silently, you have no visibility into what ran, and nothing survives a restart.\n\nfastapi-taskflow is a thin layer on top of what you already have. It does not compete with Celery, ARQ, Taskiq, or Dramatiq. It is built for teams who are already using FastAPI's native background tasks and want retries, resilience, status tracking, and a live dashboard without adding infrastructure.\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://raw.githubusercontent.com/Attakay78/fastapi-taskflow/main/docs/assets/images/dashboard.png\" target=\"_blank\"\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/Attakay78/fastapi-taskflow/main/docs/assets/images/dashboard.png\" alt=\"Task dashboard overview\" width=\"100%\" /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://raw.githubusercontent.com/Attakay78/fastapi-taskflow/main/docs/assets/images/logs.png\" target=\"_blank\"\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/Attakay78/fastapi-taskflow/main/docs/assets/images/logs.png\" alt=\"Task logs panel\" width=\"49%\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://raw.githubusercontent.com/Attakay78/fastapi-taskflow/main/docs/assets/images/error.png\" target=\"_blank\"\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/Attakay78/fastapi-taskflow/main/docs/assets/images/error.png\" alt=\"Task error and stack trace panel\" width=\"49%\" /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n## Features\n\n- Automatic retries with configurable delay and exponential backoff\n- Task IDs and full lifecycle tracking: `PENDING`, `RUNNING`, `SUCCESS`, `FAILED`, `INTERRUPTED`\n- Live admin dashboard over SSE at `/tasks/dashboard`\n- SQLite persistence out of the box, Redis as an optional extra\n- Pending task requeue: unfinished tasks at shutdown are re-dispatched on startup\n- `requeue_on_interrupt`: opt-in requeue for idempotent tasks interrupted mid-execution\n- Idempotency keys: prevent duplicate execution of the same logical operation\n- Multi-instance support: atomic requeue claiming, shared task history across instances\n- `task_log(message, level=, **extra)`: structured log entries with level filtering and arbitrary extra fields\n- `get_task_context()`: access task metadata (task_id, attempt, tags) from any code path inside a running task\n- Tags: attach key/value labels at enqueue time, forwarded to every log and lifecycle event\n- Pluggable observers: `FileLogger`, `StdoutLogger`, `InMemoryLogger`, and custom `TaskObserver` implementations\n- Argument encryption: Fernet-based at-rest encryption for task args and kwargs\n- Trace context propagation: OpenTelemetry spans flow from the request into background execution (Python 3.11+)\n- Concurrency controls: opt-in semaphore for async tasks and dedicated thread pool for sync tasks\n- Scheduled tasks: `@task_manager.schedule(every=)` and `cron=` with distributed lock for multi-instance\n- Zero-migration injection: keep your existing `BackgroundTasks` annotations\n- Both sync and async task functions supported\n\n## Installation\n\n```bash\npip install fastapi-taskflow\n```\n\nWith all optional dependencies:\n\n```bash\npip install \"fastapi-taskflow[all]\"\n```\n\nOr install only what you need:\n\n| Extra | Installs | Required for |\n|-------|----------|--------------|\n| `redis` | `redis[asyncio]` | Redis persistence backend |\n| `scheduler` | `croniter` | Cron-based scheduled tasks |\n| `encryption` | `cryptography` | Argument encryption at rest |\n\n```bash\npip install \"fastapi-taskflow[redis]\"\npip install \"fastapi-taskflow[scheduler]\"\npip install \"fastapi-taskflow[encryption]\"\n```\n\n## Quick start\n\n```python\nfrom fastapi import BackgroundTasks, FastAPI\nfrom fastapi_taskflow import TaskAdmin, TaskManager\n\ntask_manager = TaskManager(snapshot_db=\"tasks.db\", snapshot_interval=30.0)\napp = FastAPI()\n\n# auto_install=True patches FastAPI's BackgroundTasks injection so existing\n# route signatures work without any changes.\nTaskAdmin(app, task_manager, auto_install=True)\n\n\n@task_manager.task(retries=3, delay=1.0, backoff=2.0)\ndef send_email(address: str) -\u003e None:\n    ...\n\n\n@app.post(\"/signup\")\ndef signup(email: str, background_tasks: BackgroundTasks):\n    task_id = background_tasks.add_task(send_email, address=email)\n    return {\"task_id\": task_id}\n```\n\n```bash\nuvicorn examples.basic_app:app --reload\n\ncurl -X POST \"http://localhost:8000/signup?email=user@example.com\"\ncurl \"http://localhost:8000/tasks\"\ncurl \"http://localhost:8000/tasks/metrics\"\nopen \"http://localhost:8000/tasks/dashboard\"\n```\n\n## Injection patterns\n\nThree ways to get a `ManagedBackgroundTasks` instance into your routes:\n\n```python\n# Pattern 1: keep the native annotation (requires auto_install=True)\ndef route(background_tasks: BackgroundTasks):\n    task_id = background_tasks.add_task(my_func, arg)\n\n# Pattern 2: explicit managed type (also requires auto_install=True)\nfrom fastapi_taskflow import ManagedBackgroundTasks\n\ndef route(background_tasks: ManagedBackgroundTasks):\n    task_id = background_tasks.add_task(my_func, arg)\n\n# Pattern 3: explicit Depends — no install() required\nfrom fastapi import Depends\n\ndef route(tasks=Depends(task_manager.get_tasks)):\n    task_id = tasks.add_task(my_func, arg)\n```\n\n## Decorator options\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `retries` | `int` | `0` | Additional attempts after the first failure |\n| `delay` | `float` | `0.0` | Seconds before the first retry |\n| `backoff` | `float` | `1.0` | Multiplier applied to `delay` on each retry |\n| `persist` | `bool` | `False` | Save this task for requeue on restart |\n| `name` | `str` | function name | Override the name shown in the dashboard |\n| `requeue_on_interrupt` | `bool` | `False` | Requeue this task if it was mid-execution at shutdown. Only set for idempotent tasks. |\n\n## Idempotency keys\n\nPass an `idempotency_key` to `add_task()` to prevent the same logical operation from running twice. If a non-failed task with the same key already exists, the original `task_id` is returned and the task is not enqueued again.\n\n```python\ntask_id = tasks.add_task(send_notification, order_id, idempotency_key=\"order-42-notified\")\n```\n\nUseful for handling retried HTTP requests, duplicate webhook deliveries, or double-clicks.\n\n## API endpoints\n\n| Method | Path | Description |\n|--------|------|-------------|\n| `GET` | `/tasks` | List all tasks |\n| `GET` | `/tasks/{task_id}` | Single task detail |\n| `GET` | `/tasks/metrics` | Aggregated stats |\n| `GET` | `/tasks/dashboard` | Live HTML dashboard |\n| `POST` | `/tasks/{task_id}/retry` | Retry a failed or interrupted task |\n\n## Multi-instance deployments\n\nfastapi-taskflow supports running multiple instances behind a load balancer when a shared backend is configured.\n\n**Same host, multiple processes** -- use SQLite. All instances share the same file. Requeue claiming is atomic so only one instance picks up each task on restart.\n\n**Multiple hosts** -- use Redis. All instances share the same Redis instance. Idempotency keys, requeue claiming, and completed task history all work across hosts.\n\n```python\nfrom fastapi_taskflow.backends import RedisBackend\n\ntask_manager = TaskManager(\n    snapshot_backend=RedisBackend(\"redis://localhost:6379/0\"),\n    requeue_pending=True,\n)\n```\n\n**Dashboard in multi-instance deployments** -- the dashboard shows live tasks for the instance it is connected to. Completed tasks from all instances are visible via the shared backend (with a short cache window). For accurate live task visibility, route dashboard traffic to a single instance using sticky sessions at the load balancer. See the [multi-instance guide](docs/guide/multi-instance.md) for examples.\n\n**Known caveats:**\n- Live `PENDING` and `RUNNING` tasks from other instances are not visible in real time. Each instance only holds its own in-memory state.\n- SQLite multi-instance only works when all processes share the same file path on the same host. It does not work across separate machines.\n- Tasks in `RUNNING` state at the time of a hard crash (SIGKILL, OOM) cannot be recovered. Only clean shutdowns trigger the pending store write.\n\n## Structured task logging\n\n`task_log()` accepts an optional `level=` and arbitrary `**extra` keyword fields. Extras are forwarded to observers as structured fields rather than embedded in the message string.\n\n```python\nfrom fastapi_taskflow import get_task_context, task_log\n\n@task_manager.task(retries=3)\ndef process_order(order_id: int) -\u003e None:\n    ctx = get_task_context()\n    task_log(\"Processing order\", order_id=order_id, attempt=ctx.attempt if ctx else 0)\n    task_log(\"Payment gateway error\", level=\"warning\", order_id=order_id, code=503)\n```\n\n`get_task_context()` returns a `TaskContext` with `task_id`, `func_name`, `attempt`, and `tags` from any code path inside a running task.\n\n## Observability\n\nPass one or more observers to `loggers=` to receive structured `LogEvent` and `LifecycleEvent` objects for every `task_log()` call and status transition:\n\n```python\nfrom fastapi_taskflow import FileLogger, StdoutLogger, TaskManager\n\ntask_manager = TaskManager(\n    snapshot_db=\"tasks.db\",\n    loggers=[\n        FileLogger(\"tasks.log\", log_lifecycle=True),\n        StdoutLogger(log_lifecycle=True, min_level=\"warning\"),\n    ],\n)\n```\n\nBuilt-in observers:\n\n| Observer | Description |\n|----------|-------------|\n| `FileLogger` | Rotating plain text file. Works with `tail -f`, `grep`, and log shippers. |\n| `StdoutLogger` | Prints to stdout. Suitable for containers with a log agent. |\n| `InMemoryLogger` | Captures events in memory for test assertions. |\n\nThe `log_file` shorthand on `TaskManager` remains fully supported and creates a `FileLogger` internally.\n\n## Tags\n\nAttach key/value labels to a task at enqueue time. Tags flow through to every log and lifecycle event.\n\n```python\ntask_id = tasks.add_task(\n    send_email,\n    address=email,\n    tags={\"user_id\": str(user_id), \"source\": \"signup\"},\n)\n```\n\n## Argument encryption\n\nWhen tasks carry sensitive data, encrypt args and kwargs at rest with Fernet:\n\n```python\nimport os\nfrom fastapi_taskflow import TaskManager\n\ntask_manager = TaskManager(\n    snapshot_db=\"tasks.db\",\n    encrypt_args_key=os.environ[\"TASK_ENCRYPTION_KEY\"],\n)\n```\n\nGenerate a key:\n\n```bash\npython -c \"from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())\"\n```\n\nArgs and kwargs are encrypted at `add_task()` time and decrypted only inside the executor. They are never stored in plain text in the task store, the database, or any log file. Requires `pip install \"fastapi-taskflow[encryption]\"`.\n\n## Concurrency controls\n\nBy default, tasks share the event loop and thread pool with request handlers. Under burst task load this can increase request latency. Both controls are opt-in and do not change existing behaviour when not set.\n\n**`max_concurrent_tasks`** — caps how many async tasks hold event loop time simultaneously via an `asyncio.Semaphore`. Tasks waiting for a slot are parked without blocking the loop.\n\n**`max_sync_threads`** — runs sync task functions in a dedicated `ThreadPoolExecutor`, isolated from the default pool used by sync request handlers.\n\n```python\ntask_manager = TaskManager(\n    snapshot_db=\"tasks.db\",\n    max_concurrent_tasks=10,\n    max_sync_threads=8,\n)\n```\n\nBoth default to `None`. When not set, execution is identical to previous versions.\n\n## Scheduled tasks\n\nRun functions automatically at a fixed interval or on a cron expression. Scheduled tasks go through the same execution path as manually enqueued tasks, so retries, logging, persistence, and the dashboard all work without any extra setup.\n\n```python\nfrom fastapi import FastAPI\nfrom fastapi_taskflow import TaskAdmin, TaskManager, task_log\n\ntask_manager = TaskManager(snapshot_db=\"tasks.db\")\napp = FastAPI()\nTaskAdmin(app, task_manager)\n\n\n@task_manager.schedule(every=300, retries=1)\nasync def health_check() -\u003e None:\n    task_log(\"Running health check\")\n    ...\n\n\n@task_manager.schedule(cron=\"0 2 * * *\")\ndef nightly_cleanup() -\u003e None:\n    task_log(\"Starting cleanup\")\n    ...\n\n\n@task_manager.schedule(cron=\"0 9 * * *\", timezone=\"America/New_York\")\nasync def morning_report() -\u003e None:\n    ...\n```\n\nCron expressions require `pip install \"fastapi-taskflow[scheduler]\"`. Interval-based schedules have no extra dependencies. Cron expressions default to UTC. Pass any IANA timezone name with `timezone=` to evaluate in local time.\n\nScheduled tasks also appear in the task registry, so they can be enqueued manually from a route alongside their automatic schedule. In multi-instance deployments, a distributed lock ensures only one instance fires each entry per interval.\n\n## Custom dashboard title\n\nReplace the \"fastapi-taskflow\" label in the dashboard header and login page with your own app name:\n\n```python\nTaskAdmin(app, task_manager, title=\"My App\")\n```\n\n## File logging\n\nIn addition to the observer system, a plain text log file can be configured directly on `TaskManager`:\n\n```python\ntask_manager = TaskManager(\n    snapshot_db=\"tasks.db\",\n    log_file=\"tasks.log\",\n    log_lifecycle=True,\n)\n```\n\nEach line has the format `[task_id] [func_name] 2026-01-01T12:00:00 message`. For multi-process or multi-host deployments see the [file logging guide](docs/guide/file-logging.md).\n\n## What this is not\n\nfastapi-taskflow does not compete with Celery, ARQ, Taskiq, or Dramatiq. Those tools are built for distributed workers, message brokers, and high-throughput task routing. This library is for teams using FastAPI's native `BackgroundTasks` who want retries, visibility, and resilience without adding worker infrastructure.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fattakay78%2Ffastapi-taskflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fattakay78%2Ffastapi-taskflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fattakay78%2Ffastapi-taskflow/lists"}