{"id":36936896,"url":"https://github.com/vuongtlt13/aqworker","last_synced_at":"2026-01-30T11:45:53.718Z","repository":{"id":326346598,"uuid":"1103779187","full_name":"vuongtlt13/aqworker","owner":"vuongtlt13","description":"Async Redis-backed job runner for Python apps. AQWorker orchestrates workers, handlers, and queues with auto-discovery, retries, CLI tooling, and FastAPI-friendly async processing while staying framework agnostic.","archived":false,"fork":false,"pushed_at":"2025-12-10T04:20:37.000Z","size":233,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-01-13T19:56:30.575Z","etag":null,"topics":["async-await","asyncio","fastapi","job-queue","python","python3","python311","python312","redis","worker-queue"],"latest_commit_sha":null,"homepage":"https://github.com/vuongtlt13/aqworker","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vuongtlt13.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-25T10:23:24.000Z","updated_at":"2025-12-12T09:18:41.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/vuongtlt13/aqworker","commit_stats":null,"previous_names":["vuongtlt13/aqworker"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/vuongtlt13/aqworker","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vuongtlt13%2Faqworker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vuongtlt13%2Faqworker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vuongtlt13%2Faqworker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vuongtlt13%2Faqworker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vuongtlt13","download_url":"https://codeload.github.com/vuongtlt13/aqworker/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vuongtlt13%2Faqworker/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28911978,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-30T08:15:08.179Z","status":"ssl_error","status_checked_at":"2026-01-30T08:14:31.507Z","response_time":66,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async-await","asyncio","fastapi","job-queue","python","python3","python311","python312","redis","worker-queue"],"created_at":"2026-01-13T10:01:57.266Z","updated_at":"2026-01-30T11:45:53.711Z","avatar_url":"https://github.com/vuongtlt13.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AQWorker - Async Queue Worker\n\n## Overview\n\nAQWorker is a Redis-based background job processing system for Python applications. It provides a simple, efficient way to handle asynchronous tasks with support for multiple worker types, queue isolation, and job status tracking. While it works seamlessly with FastAPI, it's a standalone package that can be used with any Python application.\n\n## Features\n\n- **Framework Agnostic**: Works with FastAPI, Django, Flask, or any Python application\n- **Async/Await Support**: Built with async/await for modern Python applications\n- **Redis-based**: Uses Redis for reliable job queue management\n- **Worker Registry**: Centralized worker management with automatic discovery\n- **Handler Registry**: Organized handler system for job processing\n- **Handler Auto-Discovery**: Import handler packages automatically via `include_packages`\n- **CLI Tools**: Command-line interface for managing workers and jobs\n- **Queue Isolation**: Separate queues for different job types\n- **Job Status Tracking**: Track job lifecycle (PENDING, PROCESSING, COMPLETED, FAILED, CANCELLED)\n- **Automatic Retries**: Built-in retry mechanism with configurable delays\n\n## Architecture\n\n```\n┌─────────────┐    get_next_job()    ┌─────────────┐    dequeue()    ┌─────────────┐\n│   Worker    │ ────────────────────►│ JobService  │ ──────────────► │  JobQueue   │\n│             │                      │             │                 │             │\n│             │ ◄────────────────────│             │ ◄────────────── │             │\n└─────────────┘    complete_job()    └─────────────┘   complete_job()└─────────────┘\n\n┌─────────────┐    enqueue_job()     ┌─────────────┐    enqueue()    ┌─────────────┐\n│ Application │ ────────────────────►│   AQWorker  │ ──────────────► │  JobQueue   │\n│ (FastAPI/   │                      │             │                 │             │\n│  Django/    │                      │  - Worker   │                 │             │\n│  Flask/     │                      │    Registry │                 │             │\n│  Script)    │                      │  - Handler  │                 │             │\n│             │                      │    Registry │                 │             │\n└─────────────┘                      │  - Job      │                 └─────────────┘\n                                     │    Service  │\n                                     └─────────────┘\n```\n\n## Components\n\n### 1. AQWorker (`core.py`)\n- **Purpose**: Main orchestrator class that manages workers, handlers, and job service\n- **Features**:\n  - Worker and handler registration\n  - Worker instance creation\n  - Job service integration\n  - Centralized configuration\n\n### 2. WorkerRegistry (`worker/registry.py`)\n- **Purpose**: Manages worker class registrations\n- **Features**:\n  - Register worker classes with names\n  - List available workers\n  - Get worker definitions\n\n### 3. HandlerRegistry (`handler/registry.py`)\n- **Purpose**: Manages handler class registrations\n- **Features**:\n  - Register handler classes\n  - Lookup handlers by name\n  - Snapshot of all registered handlers\n\n### 4. JobService (`job/service.py`)\n- **Purpose**: High-level service layer for job operations\n- **Features**:\n  - Enqueue jobs\n  - Dequeue jobs for workers\n  - Get job status\n  - Queue statistics\n  - Job lifecycle management\n\n### 5. JobQueue (`job/queue.py`)\n- **Purpose**: Low-level Redis operations for job management\n- **Features**:\n  - FIFO queue processing\n  - Async/await support\n  - Job status tracking\n  - Redis operations: enqueue, dequeue, complete_job\n\n### 6. BaseWorker (`worker/base.py`)\n- **Purpose**: Base class for all workers\n- **Features**:\n  - Automatic job polling\n  - Concurrent job processing\n  - Error handling and retries\n  - Health monitoring\n\n### 7. Job \u0026 CronJob (`job/base.py`)\n- **Purpose**: Base classes for job handlers\n- **Job**: For one-time jobs\n- **CronJob**: For scheduled/recurring jobs with cron expressions\n- **Features**:\n  - Async/sync handler support\n  - Standardized job processing interface\n  - Required `queue_name` attribute\n\n## Installation\n\n```bash\npip install aqworker\n```\n\nOr install from source:\n\n```bash\ngit clone \u003crepository\u003e\ncd aqworker\npip install -e .\n```\n\n## Quick Start\n\n### 1. Define Handlers\n\n**Method 1: Using Job class (for one-time jobs)**\n\n```python\n# handlers.py\nfrom aqworker import Job\n\nclass EmailJob(Job):\n    name = \"email\"\n    queue_name = \"emails\"  # Required: specify which queue this job uses\n    \n    async def handle(self, data: dict) -\u003e bool:\n        recipient = data.get(\"recipient\")\n        subject = data.get(\"subject\")\n        body = data.get(\"body\")\n        \n        # Send email logic here\n        print(f\"Sending email to {recipient}: {subject}\")\n        return True\n```\n\n**Method 2: Using CronJob class (for scheduled jobs)**\n\n```python\nfrom aqworker import CronJob\n\nclass DailyReportCronJob(CronJob):\n    name = \"daily_report\"\n    queue_name = \"reports\"  # Required: specify which queue this job uses\n    \n    @classmethod\n    def cron(cls) -\u003e str:\n        \"\"\"Cron expression: run daily at midnight.\"\"\"\n        return \"0 0 * * *\"\n    \n    async def handle(self, data: dict) -\u003e bool:\n        # Generate daily report\n        print(\"Generating daily report...\")\n        return True\n```\n\n**Method 3: Using decorators**\n\n```python\nfrom aqworker import AQWorker\n\naq_worker = AQWorker()\n\n# Job decorator (one-time jobs)\n@aq_worker.job(name='send_email', queue_name='emails')  # queue_name is required\nasync def send_email(data: dict) -\u003e bool:\n    recipient = data.get(\"recipient\")\n    print(f\"Sending email to {recipient}\")\n    return True\n\n# CronJob decorator (scheduled jobs)\n@aq_worker.cronjob(cron='0 0 * * *', name='daily_report', queue_name='reports')  # queue_name is required\nasync def daily_report(data: dict) -\u003e bool:\n    print(\"Generating daily report...\")\n    return True\n```\n\n**Important Notes:**\n- `queue_name` is **required** for all handlers (Job, CronJob, or decorators)\n- Each handler must specify which queue it uses\n- If you override `queue_name` when enqueueing, you'll get a warning\n\n### 2. Define Workers\n\n```python\n# workers.py\nfrom aqworker import BaseWorker, WorkerConfig\n\nclass EmailWorker(BaseWorker):\n    worker_name = \"email\"\n    worker_config = WorkerConfig(\n        queue_names=[\"emails\"],\n        max_concurrent_jobs=3,\n        poll_interval=0.5,\n    )\n```\n\n### 3. Initialize AQWorker\n\n```python\n# aq_worker.py\nfrom aqworker import AQWorker\nfrom aqworker.job.service import JobService\nfrom workers import EmailWorker\nfrom handlers import EmailHandler  # optional when using include_packages\n\n# Create JobService\njob_service = JobService()\n\n# Initialize AQWorker and auto-import handlers package\naq_worker = AQWorker(include_packages=[\"handlers\"])\n\n# Register workers\naq_worker.register_worker(EmailWorker)\n# Handlers inside \"handlers\" will be discovered automatically (Job, CronJob, or decorators)\n# You can still register manually if desired:\n# aq_worker.register_handler(EmailJob)\n\n# Connect job service\naq_worker.listen(job_service)\n```\n\n#### Automatic handler discovery\n\nIf your handlers are spread across multiple modules, you can tell `AQWorker` to\nimport them automatically by passing `include_packages`. Every module inside\nthose packages will be imported once at startup, so any `@aq_worker.job` or\n`@aq_worker.cronjob` decorators (or subclasses of `Job`/`CronJob`) run and register themselves:\n\n```python\naq_worker = AQWorker(include_packages=[\n    \"my_project.workers\",\n    \"examples.simple.aq_worker\",\n])\n```\n\nYou can also call `aq_worker.autodiscover_handlers([...])` later if you need to\nload additional packages dynamically. After discovery completes AQWorker logs the\nfull list of available handlers so you can confirm everything loaded correctly.\n\n### 4. Enqueue Jobs\n\n```python\n# In your application\nfrom worker import aq_worker\n\n# Method 1: Using handler class (queue_name is taken from handler)\njob = await aq_worker.job_service.enqueue_job(\n    handler=EmailJob,  # queue_name is automatically taken from EmailJob.queue_name\n    data={\n        \"recipient\": \"user@example.com\",\n        \"subject\": \"Welcome!\",\n        \"body\": \"Welcome to our service!\"\n    }\n)\n\n# Method 2: Using handler name (queue_name must be provided)\njob = await aq_worker.job_service.enqueue_job(\n    handler=\"email\",\n    queue_name=\"emails\",  # Required when handler is a string\n    data={\n        \"recipient\": \"user@example.com\",\n        \"subject\": \"Welcome!\",\n        \"body\": \"Welcome to our service!\"\n    }\n)\n\n# Method 3: Override queue_name (will log a warning if different from handler's queue_name)\njob = await aq_worker.job_service.enqueue_job(\n    handler=EmailJob,\n    queue_name=\"other_queue\",  # Warning: overrides EmailJob.queue_name\n    data={...}\n)\n```\n\n### 5. Run Workers\n\n**Using CLI:**\n\n```bash\n# Option 1: Pass file path\naqworker start email aq_worker.py\n\n# Option 2: Use environment variable\nexport AQWORKER_FILE=aq_worker.py\naqworker start email\n```\n\n**Using Python:**\n\n```python\nimport asyncio\nfrom worker import aq_worker\n\nasync def main():\n    worker = aq_worker.create_worker(\"email\")\n    await worker.run()\n\nasyncio.run(main())\n```\n\n### 6. Run Cron Scheduler (Beat)\n\nIf you have CronJob handlers, you need to run the beat service to schedule them. The beat service automatically checks cron expressions and enqueues jobs when they match.\n\n**Using CLI:**\n\n```bash\n# Start beat service\naqworker beat aq_worker.py\n\n# With custom options\naqworker beat aq_worker.py --check-interval 0.1\n\n# Using environment variable\nexport AQWORKER_FILE=aq_worker.py\naqworker beat\n```\n\n\u003e ### 🔔 Cron expression primer  \n\u003e AQWorker supports both classic 5-field cron strings (`minute hour day month weekday`) and 6-field strings that include seconds. When we detect six fields we call `croniter(..., second_at_beginning=True)` so the **first slot is the seconds field**. Double-check your syntax:\n\u003e\n\u003e | Cadence               | Expression        | Meaning                                                   |\n\u003e |-----------------------|-------------------|-----------------------------------------------------------|\n\u003e | Every 10 seconds      | `*/10 * * * * *`  | 6-field format. `*/10` sits in the seconds field.         |\n\u003e | Every minute at 0 sec | `0 * * * * *`     | Fires once per minute exactly at the top of the minute.   |\n\u003e | Every 5 minutes       | `*/5 * * * *`     | Classic 5-field format on the minutes column.             |\n\u003e | Weekdays at 09:00     | `0 9 * * MON-FRI` | Standard cron with no seconds column.                     |\n\u003e\n\u003e **Caution:** If you accidentally supply six fields when you intended five, your job may run every few seconds instead of every few minutes. Always review the first token—when there are six tokens, it is the seconds slot.\n\n**Using Python:**\n\n```python\nfrom aqworker import AQWorker, CronScheduler\n\naq_worker = AQWorker(include_packages=[\"handlers\"])\n\n# Create and start cron scheduler\ncron_scheduler = CronScheduler(\n    handler_registry=aq_worker.handler_registry,\n    job_service=aq_worker.job_service,\n    # check_interval defaults to 0.1 seconds (supports second-level cron)\n)\n\nawait cron_scheduler.start()\n```\n\n**Important Notes:**\n- Each CronJob must have `queue_name` defined (required)\n- Beat service runs independently from workers\n- Supports both 5-field (minute-level) and 6-field (second-level) cron expressions\n- Default check interval is 0.1 seconds for better precision\n\n## Usage Examples\n\n### Standalone Python Application\n\n```python\n# main.py\nimport asyncio\nfrom worker import aq_worker\n\nasync def main():\n    # Enqueue jobs\n    for i in range(10):\n        job = await aq_worker.job_service.enqueue_job(\n            handler=\"email\",\n            queue_name=\"emails\",\n            data={\"recipient\": f\"user{i}@example.com\", \"subject\": f\"Email {i}\"}\n        )\n        print(f\"Enqueued job: {job.id}\")\n    \n    # Start aq_worker\n    worker = aq_worker.create_worker(\"email\")\n    await worker.run()\n\nasyncio.run(main())\n```\n\n### FastAPI Integration\n\nThe repo ships with a complete FastAPI + AQWorker demo under `examples/simple_fastapi/`.\nKey files:\n\n- `worker.py` – configures `AQWorker`, registers the example workers, and wires up the\n  shared `JobService`.\n- `main.py` – FastAPI app that exposes `/jobs/email`, `/jobs/notification`, queue stats,\n  and discovery endpoints.\n- `client.py` – small CLI client that calls the HTTP endpoints so you can watch jobs\n  flow through the queues.\n\nMinimal excerpt of the worker wiring:\n\n```python\n# examples/simple_fastapi/worker.py\nfrom aqworker import AQWorker\nfrom workers import EmailWorker, NotificationWorker\nfrom job_service import job_service\n\naq_worker = AQWorker(include_packages=[\"handlers\"])\naq_worker.register_worker(EmailWorker)\naq_worker.register_worker(NotificationWorker)\naq_worker.listen(job_service)\n```\n\nTo run the demo API and fire sample requests:\n\n```bash\ncd examples/simple_fastapi\nuv run python main.py            # start FastAPI\n\n# in another shell: send a job via the bundled client\nuv run python client.py --recipient user@example.com --subject Test --body \"Hello!\"\n```\n\n### Django Integration\n\n```python\n# views.py\nfrom django.http import JsonResponse\nfrom worker import aq_worker\nimport asyncio\n\nasync def send_email_view(request):\n    job = await aq_worker.job_service.enqueue_job(\n        handler=\"email\",\n        queue_name=\"emails\",\n        data={\n            \"recipient\": request.POST.get(\"recipient\"),\n            \"subject\": request.POST.get(\"subject\"),\n            \"body\": request.POST.get(\"body\")\n        }\n    )\n    return JsonResponse({\"job_id\": job.id})\n\n# Use asyncio.run or Django's async support\n```\n\n## CLI Commands\n\nAQWorker provides a CLI tool for managing workers and jobs:\n\n### List Commands\n\n```bash\n# List available workers\naqworker list:aq_worker [file_path]\n\n# List registered handlers\naqworker list:handlers [file_path]\n\n# List queues\naqworker list:queue [file_path]\n```\n\n### Worker Commands\n\n```bash\n# Start a aq_worker\naqworker start \u003cworker_name\u003e [file_path]\n\n# Get queue statistics\naqworker stats \u003cqueue_name\u003e [file_path]\n```\n\n### Using Environment Variable\n\n```bash\n# Set AQWorker file path\nexport AQWORKER_FILE=aq_worker.py\n\n# Use commands without file path\naqworker list:aq_worker\naqworker start email\naqworker stats emails\n```\n\n**Note**: The file path should point to a Python file containing an `AQWorker` instance. The CLI automatically finds the instance (looking for variables named `aq_worker`, `worker`, `aqworker`, or `aq`).\n\n## Configuration\n\n### Environment Variables\n\n```bash\n# Redis Configuration (used when creating JobService)\nREDIS_HOST=localhost\nREDIS_PORT=6379\nREDIS_DB=0\n\n# AQWorker CLI\nAQWORKER_FILE=aq_worker.py  # Path to file containing AQWorker instance\n```\n\n### Worker Configuration\n\n```python\nfrom aqworker import WorkerConfig\n\nclass MyWorker(BaseWorker):\n    worker_name = \"my_worker\"\n    worker_config = WorkerConfig(\n        queue_names=[\"my_queue\"],\n        max_concurrent_jobs=5,      # Max jobs processed simultaneously\n        poll_interval=0.5,          # Seconds between queue polls\n        job_timeout=300,            # Job timeout in seconds\n    )\n```\n\n## Job Lifecycle\n\n```\n1. ENQUEUE: Job created via JobService.enqueue_job()\n2. QUEUE: Job added to Redis queue (aqw:{queue_name})\n3. DEQUEUE: Worker gets job via JobService.get_next_job()\n4. PROCESS: Job moved to processing queue, handler executed\n5. COMPLETE: Job marked as completed/failed\n6. CLEANUP: Old jobs cleaned up automatically\n```\n\n## Queue System\n\n### Redis key prefixes\n\nAQWorker keeps every Redis key namespaced with the `aqw` prefix so multiple apps can\nshare the same Redis without collisions. The most important keys are:\n\n- `aqw:{queue_name}` – pending FIFO queue for a specific worker queue\n- `aqw:processing`, `aqw:completed`, `aqw:failed` – global processing/completed/failed lists\n- `aqw:job:{job_id}` – job status hash (timestamps, error state, etc.)\n- `aqw:jl:{job_id}` – per-job lock key used to ensure only one worker processes the job\n\nAll public APIs use helpers from `aqworker.constants` (`get_queue_name`,\n`get_job_status_key`, `get_job_lock_key`) so you rarely need to construct these strings\nmanually.\n\n### Queue Structure\n\n```\nRedis Keys:\n├── aqw:emails        # Email jobs (FIFO)\n├── aqw:notifications # Notification jobs (FIFO)\n├── aqw:processing    # Jobs currently being processed\n├── aqw:completed     # Successfully completed jobs\n└── aqw:failed        # Failed jobs\n```\n\n### Queue Isolation\n\n- Each worker processes specific queue names\n- Jobs are isolated by queue name\n- No cross-queue interference\n- Use different queues for different job types\n\n### FIFO Processing\n\n- Simple first-in-first-out queue processing\n- Jobs processed in order of arrival\n- All jobs in a queue are processed equally\n- Use different queue names for different priority levels\n\n## Advanced Usage\n\n### Custom Job Data\n\n```python\n# Using handler class (queue_name from handler)\njob = await aq_worker.job_service.enqueue_job(\n    handler=EmailJob,  # queue_name automatically taken from EmailJob.queue_name\n    data={\n        \"recipient\": \"user@example.com\",\n        \"subject\": \"Welcome\",\n        \"body\": \"Welcome!\",\n        \"attachments\": [\"file1.pdf\", \"file2.jpg\"],\n        \"priority\": \"high\"\n    },\n    metadata={\n        \"source\": \"api\",\n        \"user_id\": \"12345\",\n        \"campaign_id\": \"summer2024\"\n    },\n    max_retries=5,\n    retry_delay=60\n)\n\n# Or using handler name (queue_name required)\njob = await aq_worker.job_service.enqueue_job(\n    handler=\"email\",\n    queue_name=\"emails\",  # Required when handler is a string\n    data={...},\n    metadata={...}\n)\n```\n\n### Multiple Workers\n\n```python\n# Register multiple workers\naq_worker.register_worker(EmailWorker)\naq_worker.register_worker(NotificationWorker)\naq_worker.register_worker(ReportWorker)\n\n# Start specific aq_worker\nworker = aq_worker.create_worker(\"email\")\nawait worker.run()\n```\n\n### Handler with Async Support\n\n```python\nclass AsyncJob(Job):\n    name = \"async_task\"\n    queue_name = \"async_tasks\"  # Required\n    \n    async def handle(self, data: dict) -\u003e bool:\n        # Async operations\n        result = await some_async_operation(data)\n        return result is not None\n```\n\n## Best Practices\n\n### 1. Job Design\n- Keep jobs small and focused\n- Use descriptive handler names\n- Include all necessary data in `data` parameter\n- Use `metadata` for tracking/debugging information\n\n### 2. Worker Design\n- Implement proper error handling in handlers\n- Use appropriate concurrency levels\n- Monitor job processing time\n- Log important events\n\n### 3. Queue Management\n- Use descriptive queue names\n- Monitor queue depths\n- Clean up old jobs regularly\n- Balance load across workers\n\n### 4. Handler Design\n- **Required**: All handlers (Job, CronJob) must define `queue_name` attribute\n- Make handlers idempotent when possible\n- Handle errors gracefully\n- Return `True` on success, `False` on failure\n- Use async handlers for I/O operations\n- Use `Job` for one-time jobs, `CronJob` for scheduled/recurring jobs\n\n## Troubleshooting\n\n### Common Issues\n\n1. **Jobs not processing**\n   - Check worker is running\n   - Verify queue names match between enqueue and worker\n   - Check Redis connection\n   - Verify handler is registered\n\n2. **High memory usage**\n   - Clean up old jobs: `await job_service.cleanup_old_jobs(days=7)`\n   - Reduce queue depths\n   - Optimize job data size\n\n3. **Slow processing**\n   - Increase worker concurrency (`max_concurrent_jobs`)\n   - Optimize handler logic\n   - Check Redis performance\n   - Consider multiple workers\n\n### Debugging\n\n```python\n# Check queue statistics\nstats = await aq_worker.job_service.get_queue_stats([\"emails\"])\nprint(f\"Queue stats: {stats}\")\n\n# Get job status\njob = await aq_worker.job_service.get_job(job_id)\nprint(f\"Job status: {job.status}\")\n\n# List registered workers\nworkers = aq_worker.get_available_workers()\nprint(f\"Available workers: {workers}\")\n\n# List registered handlers\nhandlers = aq_worker.handler_registry.snapshot()\nprint(f\"Registered handlers: {list(handlers.keys())}\")\n```\n\n## Examples\n\nThe `examples/` directory contains end-to-end demos:\n\n### `examples/simple/`\n- Pure Python example with a background thread enqueuing email jobs while a worker\n  (created via `create_worker(\"email\")`) processes them.\n- Handlers live in `examples.simple.aq_worker.handlers` and are picked up through\n  `AQWorker(include_packages=[\"examples.simple.aq_worker\"])`, so you can keep job logic\n  in separate modules without manual registration.\n- Run `python -m examples.simple.main` to start the enqueue loop and worker concurrently\n  or use the CLI (`aqworker start email examples/simple/worker.py`) in separate shells.\n\n### `examples/simple_fastapi/`\n- FastAPI service exposing REST endpoints for enqueuing jobs plus health/metadata\n  routes (`/handlers`, `/workers`, `/jobs/queues/{queue}/stats`).\n- Worker definition in `examples/simple_fastapi/worker.py` registers worker classes and\n  auto-discovers handlers via `include_packages=[\"examples.simple_fastapi.handlers\"]`.\n- `examples/simple_fastapi/handlers.py` simulates real work with 0.5–2 s `asyncio.sleep`\n  delays so you can observe concurrent processing.\n- `examples/simple_fastapi/client.py` is an `httpx` script that continuously enqueues\n  email + notification jobs every ~0.2 s—perfect for smoke-testing the API while\n  watching worker logs scroll.\n\n## Contributing\n\nWhen adding new features:\n1. Follow async/await patterns\n2. Add proper error handling\n3. Update documentation\n4. Add tests\n5. Consider performance impact\n\n## License\n\nThis project is licensed under the terms specified in the LICENSE file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvuongtlt13%2Faqworker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvuongtlt13%2Faqworker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvuongtlt13%2Faqworker/lists"}