{"id":34623700,"url":"https://github.com/block/agent-task-queue","last_synced_at":"2026-01-23T21:01:24.212Z","repository":{"id":328859687,"uuid":"1117117873","full_name":"block/agent-task-queue","owner":"block","description":"Local task queuing for AI agents. Prevents multiple agents from running expensive operations concurrently and thrashing your machine.","archived":false,"fork":false,"pushed_at":"2026-01-14T18:43:17.000Z","size":118,"stargazers_count":14,"open_issues_count":1,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-18T02:08:43.670Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/block.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":"GOVERNANCE.md","roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-15T21:25:06.000Z","updated_at":"2026-01-17T22:05:33.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/block/agent-task-queue","commit_stats":null,"previous_names":["block/agent-task-queue"],"tags_count":5,"template":false,"template_full_name":"block/oss-project-template","purl":"pkg:github/block/agent-task-queue","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/block%2Fagent-task-queue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/block%2Fagent-task-queue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/block%2Fagent-task-queue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/block%2Fagent-task-queue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/block","download_url":"https://codeload.github.com/block/agent-task-queue/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/block%2Fagent-task-queue/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28700321,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-23T17:25:48.045Z","status":"ssl_error","status_checked_at":"2026-01-23T17:25:47.153Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-24T15:43:11.075Z","updated_at":"2026-01-23T21:01:24.095Z","avatar_url":"https://github.com/block.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Agent Task Queue\n\n[![CI](https://github.com/block/agent-task-queue/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/block/agent-task-queue/actions/workflows/ci.yml)\n[![PyPI version](https://img.shields.io/pypi/v/agent-task-queue)](https://pypi.org/project/agent-task-queue/)\n[![GitHub release](https://img.shields.io/github/v/release/block/agent-task-queue)](https://github.com/block/agent-task-queue/releases)\n\n**Local task queuing for AI agents.** Prevents multiple agents from running expensive operations concurrently and thrashing your machine.\n\n## The Problem\n\nWhen multiple AI agents work on the same machine, they independently trigger expensive operations. Running these concurrently causes:\n\n- 5-minute builds stretching to 30+ minutes\n- Memory thrashing and disk I/O saturation\n- Machine unresponsiveness\n- Agents unable to coordinate with each other\n\n## How It Works\n\n**Default: Global queue** - All `run_task` calls share one queue.\n\n```\n# Agent A runs:\nrun_task(\"./gradlew test\", working_directory=\"/project\")\n\n# Agent B runs (waits for A to finish, then executes):\nrun_task(\"./gradlew build\", working_directory=\"/project\")\n```\n\n**Custom queues** - Use `queue_name` to isolate workloads:\n\n```\n# These run in separate queues (can run in parallel):\nrun_task(\"./gradlew build\", queue_name=\"android\", ...)\nrun_task(\"npm run build\", queue_name=\"web\", ...)\n```\n\nBoth agents block until their respective builds complete. The server handles sequencing automatically.\n\n## Demo: Two Agents, One Build Queue\n\n**Terminal A** - First agent requests an Android build:\n```\n\u003e Build the Android app\n\n⏺ agent-task-queue - run_task (MCP)\n  command: \"./gradlew assembleDebug\"\n  working_directory: \"/path/to/android-project\"\n\n  ⎿  \"SUCCESS exit=0 192.6s output=/tmp/agent-task-queue/output/task_1.log\"\n\n⏺ Build completed successfully in 192.6s.\n```\n\n**Terminal B** - Second agent requests the same build (started 2 seconds after A):\n```\n\u003e Build the Android app\n\n⏺ agent-task-queue - run_task (MCP)\n  command: \"./gradlew assembleDebug\"\n  working_directory: \"/path/to/android-project\"\n\n  ⎿  \"SUCCESS exit=0 32.6s output=/tmp/agent-task-queue/output/task_2.log\"\n\n⏺ Build completed successfully in 32.6s.\n```\n\n**What happened behind the scenes:**\n\n| Time | Agent A | Agent B |\n|------|---------|---------|\n| 0:00 | Started build | |\n| 0:02 | Building... | Entered queue, waiting |\n| 3:12 | **Completed** (192.6s) | Started build |\n| 3:45 | | **Completed** (32.6s) |\n\n**Why this matters:**\n\nWithout the queue, both builds would run simultaneously—fighting for CPU, memory, and disk I/O. Each build might take 5+ minutes, and your machine would be unresponsive.\n\nWith the queue:\n- **Agent B automatically waited** for Agent A to finish\n- **Agent B's build was 6x faster** (32s vs 193s) because Gradle reused cached artifacts\n- **Total time: 3:45** instead of 10+ minutes of thrashing\n- **Your machine stayed responsive** throughout\n\n## Key Features\n\n- **FIFO Queuing**: Strict first-in-first-out ordering\n- **No Queue Timeouts**: MCP keeps connection alive while waiting in queue. The `timeout_seconds` parameter only applies to execution time—tasks can wait in queue indefinitely without timing out. (see [Why MCP?](#why-mcp-instead-of-a-cli-tool))\n- **Environment Variables**: Pass `env_vars=\"ANDROID_SERIAL=emulator-5560\"`\n- **Multiple Queues**: Isolate different workloads with `queue_name`\n- **Zombie Protection**: Detects dead processes, kills orphans, clears stale locks\n- **Auto-Kill**: Tasks running \u003e 120 minutes are terminated\n\n## Installation\n\n```bash\nuvx agent-task-queue@latest\n```\n\nThat's it. [uvx](https://docs.astral.sh/uv/guides/tools/) runs the package directly from PyPI—no clone, no install, no virtual environment.\n\n## Agent Configuration\n\nAgent Task Queue works with any AI coding tool that supports MCP. Add this config to your MCP client:\n\n```json\n{\n  \"mcpServers\": {\n    \"agent-task-queue\": {\n      \"command\": \"uvx\",\n      \"args\": [\"agent-task-queue@latest\"]\n    }\n  }\n}\n```\n\n### MCP Client Configuration\n\n\u003cdetails\u003e\n\u003csummary\u003eAmp\u003c/summary\u003e\n\nInstall via CLI:\n\n```bash\namp mcp add agent-task-queue -- uvx agent-task-queue@latest\n```\n\nOr add to `.amp/settings.json` (workspace) or global settings. See [Amp Manual](https://ampcode.com/manual) for details.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eClaude Code\u003c/summary\u003e\n\nInstall via CLI (\u003ca href=\"https://docs.anthropic.com/en/docs/claude-code/mcp\"\u003eguide\u003c/a\u003e):\n\n```bash\nclaude mcp add agent-task-queue -- uvx agent-task-queue@latest\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eClaude Desktop\u003c/summary\u003e\n\nConfig file locations:\n- **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json`\n- **Linux**: `~/.config/Claude/claude_desktop_config.json`\n\nUse the standard config above.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eCline\u003c/summary\u003e\n\nOpen the MCP Servers panel \u003e Configure \u003e \"Configure MCP Servers\" to edit `cline_mcp_settings.json`. Use the standard config above.\n\nSee [Cline MCP docs](https://docs.cline.bot/mcp/configuring-mcp-servers) for details.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eCopilot / VS Code\u003c/summary\u003e\n\nRequires VS Code 1.102+ with GitHub Copilot Chat extension.\n\nConfig file locations:\n- **Workspace**: `.vscode/mcp.json`\n- **Global**: Via Command Palette \u003e \"MCP: Open User Configuration\"\n\n```json\n{\n  \"servers\": {\n    \"agent-task-queue\": {\n      \"type\": \"stdio\",\n      \"command\": \"uvx\",\n      \"args\": [\"agent-task-queue@latest\"]\n    }\n  }\n}\n```\n\nSee [VS Code MCP docs](https://code.visualstudio.com/docs/copilot/chat/mcp-servers) for details.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eCursor\u003c/summary\u003e\n\nGo to `Cursor Settings` \u003e `MCP` \u003e `+ Add new global MCP server`. Use the standard config above.\n\nConfig file locations:\n- **Global**: `~/.cursor/mcp.json`\n- **Project**: `.cursor/mcp.json`\n\nSee [Cursor MCP docs](https://docs.cursor.com/context/model-context-protocol) for details.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eFirebender\u003c/summary\u003e\n\nAdd to `firebender.json` in project root, or use Plugin Settings \u003e MCP section. Use the standard config above.\n\nSee [Firebender MCP docs](https://docs.firebender.com/context/mcp) for details.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eWindsurf\u003c/summary\u003e\n\nConfig file location: `~/.codeium/windsurf/mcp_config.json`\n\nOr use Windsurf Settings \u003e Cascade \u003e Manage MCPs. Use the standard config above.\n\nSee [Windsurf MCP docs](https://docs.windsurf.com/windsurf/cascade/mcp) for details.\n\n\u003c/details\u003e\n\n## Usage\n\nAgents use the `run_task` MCP tool for expensive operations:\n\n**Build Tools:** gradle, bazel, make, cmake, mvn, cargo build, go build, npm/yarn/pnpm build\n\n**Container Operations:** docker build, docker-compose, podman, kubectl, helm\n\n**Test Suites:** pytest, jest, mocha, rspec\n\n\u003e **Note:** Some agents automatically prefer MCP tools (Amp, Copilot, Windsurf). Others may need [configuration](#agent-configuration-notes) to prefer `run_task` over built-in shell commands.\n\n### Tool Parameters\n\n| Parameter | Required | Description |\n|-----------|----------|-------------|\n| `command` | Yes | Shell command to execute |\n| `working_directory` | Yes | Absolute path to run from |\n| `queue_name` | No | Queue identifier (default: \"global\") |\n| `timeout_seconds` | No | Max **execution** time before kill (default: 1200). Queue wait time doesn't count. |\n| `env_vars` | No | Environment variables: `\"KEY=val,KEY2=val2\"` |\n\n### Example\n\n```\nrun_task(\n    command=\"./gradlew connectedAndroidTest\",\n    working_directory=\"/project\",\n    queue_name=\"android\",\n    env_vars=\"ANDROID_SERIAL=emulator-5560\"\n)\n```\n\n### Agent Configuration Notes\n\nSome agents need additional configuration to use the queue instead of built-in shell commands.\n\n| Agent | Extra Setup | Notes |\n|-------|-------------|-------|\n| Amp, Copilot, Windsurf | ❌ None | Works out of the box |\n| **Claude Code, Cursor** | ✅ Required | Must remove Bash allowed rules |\n| Cline, Firebender | ⚠️ Maybe | Check agent docs |\n\n\u003e [!IMPORTANT]\n\u003e **Claude Code users:** If you have allowed rules like `Bash(gradle:*)` or `Bash(./gradlew:*)`, the agent will use Bash directly and **bypass the queue entirely**. You must remove these rules for the queue to work.\n\u003e\n\u003e Check both `settings.json` and `settings.local.json` (project and global) for rules like:\n\u003e - `Bash(gradle:*)`, `Bash(./gradlew:*)`, `Bash(ANDROID_SERIAL=* ./gradlew:*)`\n\u003e - `Bash(docker build:*)`, `Bash(pytest:*)`, etc.\n\u003e\n\u003e See [Claude Code setup guide](examples/claude-code/SETUP.md) for the full fix.\n\n#### Quick Agent Setup\n\nAfter installing the MCP server, tell your agent:\n\n```\n\"Configure agent-task-queue - use examples/\u003cagent-name\u003e/SETUP.md if available\"\n```\n\n**Available setup guides:**\n- [Claude Code setup](examples/claude-code/SETUP.md) - 3-step configuration\n- [Other agents](examples/) - Contributions welcome!\n\n## Configuration\n\nThe server supports the following command-line options:\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `--data-dir` | `/tmp/agent-task-queue` | Directory for database and logs |\n| `--max-log-size` | `5` | Max metrics log size in MB before rotation |\n| `--max-output-files` | `50` | Number of task output files to retain |\n| `--tail-lines` | `50` | Lines of output to include on failure |\n| `--lock-timeout` | `120` | Minutes before stale locks are cleared |\n\nPass options via the `args` property in your MCP config:\n\n```json\n{\n  \"mcpServers\": {\n    \"agent-task-queue\": {\n      \"command\": \"uvx\",\n      \"args\": [\n        \"agent-task-queue@latest\",\n        \"--max-output-files=100\",\n        \"--lock-timeout=60\"\n      ]\n    }\n  }\n}\n```\n\nRun `uvx agent-task-queue@latest --help` to see all options.\n\n## Architecture\n\n```mermaid\nflowchart TD\n    A[AI Agent\u003cbr/\u003eClaude, Cursor, Windsurf, etc.] --\u003e|MCP Protocol| B[task_queue.py\u003cbr/\u003eFastMCP Server]\n    B --\u003e|Query/Update| C[(SQLite Queue\u003cbr/\u003e/tmp/agent-task-queue/queue.db)]\n    B --\u003e|Execute| D[Subprocess\u003cbr/\u003egradle, docker, etc.]\n\n    D -.-\u003e|stdout/stderr| B\n    B -.-\u003e|blocks until complete| A\n```\n\n### Data Directory\n\nAll data is stored in `/tmp/agent-task-queue/` by default:\n- `queue.db` - SQLite database for queue state\n- `agent-task-queue-logs.json` - JSON metrics log (NDJSON format)\n\nTo use a different location, pass `--data-dir=/path/to/data` or set the `TASK_QUEUE_DATA_DIR` environment variable.\n\n### Database Schema\n\nThe queue state is stored in SQLite at `/tmp/agent-task-queue/queue.db`:\n\n| Column | Type | Description |\n|--------|------|-------------|\n| `id` | INTEGER | Auto-incrementing primary key |\n| `queue_name` | TEXT | Queue identifier (e.g., \"global\", \"android\") |\n| `status` | TEXT | Task state: \"waiting\" or \"running\" |\n| `pid` | INTEGER | MCP server process ID (for liveness check) |\n| `child_pid` | INTEGER | Subprocess ID (for orphan cleanup) |\n| `created_at` | TIMESTAMP | When task was queued |\n| `updated_at` | TIMESTAMP | Last status change |\n\n### Zombie Protection\n\nIf an agent crashes while a task is running:\n1. The next task detects the dead parent process (via PID check)\n2. It kills any orphaned child process (the actual build)\n3. It clears the stale lock\n4. Execution continues normally\n\n### Metrics Logging\n\nAll queue events are logged to `agent-task-queue-logs.json` in NDJSON format (one JSON object per line):\n\n```json\n{\"event\":\"task_queued\",\"timestamp\":\"2025-12-12T16:01:34\",\"task_id\":8,\"queue_name\":\"global\",\"pid\":23819}\n{\"event\":\"task_started\",\"timestamp\":\"2025-12-12T16:01:34\",\"task_id\":8,\"queue_name\":\"global\",\"wait_time_seconds\":0.0}\n{\"event\":\"task_completed\",\"timestamp\":\"2025-12-12T16:02:05\",\"task_id\":8,\"queue_name\":\"global\",\"command\":\"./gradlew build\",\"exit_code\":0,\"duration_seconds\":31.2,\"stdout_lines\":45,\"stderr_lines\":2}\n```\n\n**Events logged:**\n- `task_queued` - Task entered the queue\n- `task_started` - Task acquired lock and began execution\n- `task_completed` - Task finished (includes exit code and duration)\n- `task_timeout` - Task killed after timeout\n- `task_error` - Task failed with exception\n- `zombie_cleared` - Stale lock was cleaned up\n\nThe log file rotates when it exceeds 5MB (keeps one backup as `.json.1`).\n\n### Task Output Logs\n\nTo reduce token usage, full command output is written to files instead of returned directly:\n\n```\n/tmp/agent-task-queue/output/\n├── task_1.log\n├── task_2.log\n└── ...\n```\n\n**On success**, the tool returns a single line:\n```\nSUCCESS exit=0 31.2s output=/tmp/agent-task-queue/output/task_8.log\n```\n\n**On failure**, the last 50 lines of output are included:\n```\nFAILED exit=1 12.5s output=/tmp/agent-task-queue/output/task_9.log\n[error output here]\n```\n\n**Automatic cleanup**: Old files are deleted when count exceeds 50 (configurable via `MAX_OUTPUT_FILES`).\n\n**Manual cleanup**: Use the `clear_task_logs` tool to delete all output files.\n\n## CLI Tool\n\nThe `tq` command lets you run commands through the queue and inspect queue status.\n\n### Install CLI\n\n```bash\nuv tool install agent-task-queue\n```\n\nThis installs both the MCP server and the `tq` CLI persistently.\n\n### Running Commands\n\nRun commands through the same queue that agents use:\n\n```bash\ntq ./gradlew assembleDebug          # Run a build through the queue\ntq npm run build                    # Any command works\ntq -q android ./gradlew test        # Use a specific queue\ntq -t 600 npm test                  # Custom timeout (seconds)\ntq -C /path/to/project make         # Set working directory\n```\n\nThis prevents resource contention between you and AI agents - when you run a build via `tq`, any agent-initiated builds will wait in the same queue.\n\n### Inspecting the Queue\n\n```bash\ntq list              # Show current queue\ntq logs              # Show recent activity\ntq logs -n 50        # Show last 50 entries\ntq clear             # Clear stuck tasks\ntq --data-dir PATH   # Use custom data directory\n```\n\nRespects `TASK_QUEUE_DATA_DIR` environment variable.\n\n\u003e **Note:** Without installing, you can run one-off commands with:\n\u003e ```bash\n\u003e uvx --from agent-task-queue tq list\n\u003e ```\n\n## Troubleshooting\n\n### Tasks stuck in queue\n\n```bash\ntq list    # Check queue status\ntq clear   # Clear all tasks\n```\n\n### \"Database is locked\" errors\n\n```bash\nps aux | grep task_queue                  # Check for zombie processes\nrm -rf /tmp/agent-task-queue/             # Delete and restart\n```\n\n### Server not connecting\n\n1. Ensure `uvx` is in your PATH (install [uv](https://github.com/astral-sh/uv) if needed)\n2. Test manually: `uvx agent-task-queue@latest`\n\n## Development\n\nFor contributors:\n\n```bash\ngit clone https://github.com/block/agent-task-queue.git\ncd agent-task-queue\nuv sync                      # Install dependencies\nuv run pytest -v             # Run tests\nuv run python task_queue.py  # Run server locally\n```\n\n## Platform Support\n\n- macOS\n- Linux\n\n## Why MCP Instead of a CLI Tool?\n\nThe first attempt at solving this problem was a file-based queue CLI that wrapped commands:\n\n```bash\nqueue-cli ./gradlew build\n```\n\n**The fatal flaw:** AI tools have built-in shell timeouts (30s-120s). If a job waited in queue longer than the timeout, the agent gave up—even though the job would eventually run.\n\n```mermaid\nflowchart LR\n    subgraph cli [CLI Approach]\n        A1[Agent] --\u003e B1[Shell]\n        B1 --\u003e C1[CLI]\n        C1 --\u003e D1[Queue]\n        B1 -.-\u003e |\"⏱️ TIMEOUT!\"| A1\n    end\n\n    subgraph mcp [MCP Approach]\n        A2[Agent] --\u003e |MCP Protocol| B2[Server]\n        B2 --\u003e C2[Queue]\n        B2 -.-\u003e |\"✓ blocks until complete\"| A2\n    end\n```\n\n**Why MCP solves this:**\n- The MCP server keeps the connection alive indefinitely\n- The agent's tool call blocks until the task completes\n- No timeout configuration needed—it \"just works\"\n- The server manages the queue; the agent just waits\n\n| Aspect | CLI Wrapper | Agent Task Queue |\n|--------|-------------|----------------|\n| Timeout handling | External workarounds | Solved by design |\n| Queue storage | Filesystem | SQLite (WAL mode) |\n| Integration | Wrap every command | Automatic tool selection |\n| Agent compatibility | Varies by tool | Universal |\n\n## License\n\nApache 2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblock%2Fagent-task-queue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fblock%2Fagent-task-queue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblock%2Fagent-task-queue/lists"}