{"id":51317577,"url":"https://github.com/arc-language/arc-jobs","last_synced_at":"2026-07-01T09:01:14.567Z","repository":{"id":361414446,"uuid":"1254376051","full_name":"arc-language/arc-jobs","owner":"arc-language","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-30T13:57:28.000Z","size":50,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-30T15:22:07.139Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arc-language.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-30T13:40:47.000Z","updated_at":"2026-05-30T13:57:32.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/arc-language/arc-jobs","commit_stats":null,"previous_names":["arc-language/arc-jobs"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/arc-language/arc-jobs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arc-language%2Farc-jobs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arc-language%2Farc-jobs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arc-language%2Farc-jobs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arc-language%2Farc-jobs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arc-language","download_url":"https://codeload.github.com/arc-language/arc-jobs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arc-language%2Farc-jobs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34999792,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-01T02:00:05.325Z","response_time":130,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-01T09:01:13.510Z","updated_at":"2026-07-01T09:01:14.555Z","avatar_url":"https://github.com/arc-language.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# @arc-lang/arc-jobs\n\n[![npm](https://img.shields.io/npm/v/@arc-lang/arc-jobs?color=6366f1\u0026label=npm)](https://www.npmjs.com/package/@arc-lang/arc-jobs)\n[![CI](https://github.com/arc-language/arc-jobs/actions/workflows/test.yml/badge.svg)](https://github.com/arc-language/arc-jobs/actions/workflows/test.yml)\n[![License: MIT](https://img.shields.io/badge/license-MIT-22c55e.svg)](LICENSE)\n[![Bun compatible](https://img.shields.io/badge/bun-%E2%89%A51.0-f6dece.svg)](https://bun.sh)\n\nProduction background jobs for [Arc](https://arc-language.dev) — SQLite, Redis, and memory adapters with cron scheduling, deduplication, progress tracking, and a live admin dashboard.\n\n---\n\n## Why arc-jobs?\n\n- **Zero broker in development** — SQLite queue runs in-process; no Redis to set up locally\n- **Compile-time type safety** — wrong job signatures fail at `arc build`, not in production\n- **No separate worker process** — jobs run inside your server, one fewer thing to deploy\n- **`@unique` deduplication** — prevents duplicate jobs while one is already pending or running (celery-once built-in)\n- **`@schedule` cron** — replaces a separate Celery Beat / BullMQ scheduler process\n\n---\n\n## Contents\n\n- [Install](#install)\n- [Quick Start](#quick-start)\n- [Annotations Reference](#annotations-reference)\n- [Calling Jobs](#calling-jobs)\n- [Queue Adapters](#queue-adapters)\n- [Comparison](#comparison)\n- [Arc Ecosystem](#arc-ecosystem)\n- [Admin Dashboard](#admin-dashboard)\n- [CLI](#cli)\n- [Testing](#testing)\n- [Cloudflare Workers](#cloudflare-workers)\n- [Contributing](#contributing)\n- [License](#license)\n\n---\n\n## Install\n\n```bash\nbun add @arc-lang/arc-jobs\n```\n\nAdd queue configuration to `arc.config.json`:\n\n```json\n{\n  \"queues\": {\n    \"default\":   { \"backend\": \"sqlite\" },\n    \"payments\":  { \"backend\": \"redis\", \"url\": \"${REDIS_URL}\" },\n    \"reports\":   { \"backend\": \"sqlite\", \"timeout\": 300000 }\n  }\n}\n```\n\nOr run the interactive setup:\n\n```bash\narc-jobs init\n```\n\n---\n\n## Quick Start\n\n### 1. Define jobs in your `.arc` server files\n\n```arc\n// server/jobs.arc\n\n// Basic job — uses the \"default\" queue\njob SendWelcomeEmail(userId: Int)\n  const user = db.users.find(userId)\n  email.send({ to: user.email, subject: \"Welcome!\" })\n\n// High-priority job on a dedicated Redis queue\n@queue payments\n@priority high\n@retries 5\njob ProcessPayment(orderId: Int, amount: Float)\n  // ... payment logic\n\n// Prevent duplicate sends while job is running or pending\n@queue notifications\n@unique timeout=3600000 strategy=skip\njob SendInvoice(invoiceId: Int)\n  email.send({ to: \"...\" })\n\n// Schedule without a separate process\n@schedule \"0 9 * * 1\"\njob WeeklyReport()\n  // ... runs every Monday at 9am\n```\n\n### 2. Call jobs from routes\n\n```arc\n@route post \"/orders\" -\u003e Response\n  const order = db.orders.create(parseBody(request))\n  ProcessPayment(order.id, order.total)            // fire-and-forget\n  SendWelcomeEmail.delay(86400000, order.userId)   // delayed 24h\n  json(order, 201)\n```\n\n### 3. Build and run\n\n```bash\narc build-server .\nbun dist/server.js\n```\n\nNo broker to set up. Jobs run inside your server process.\n\n---\n\n## Annotations Reference\n\n### `@queue \u003cname\u003e`\n\nRoute the job to a named queue from `arc.config.json`. Omitting `@queue` uses the `\"default\"` queue.\n\n```arc\n@queue payments\njob ProcessPayment(orderId: Int)\n  // uses the \"payments\" queue (Redis, in this example)\n```\n\n### `@schedule \"\u003ccron\u003e\"`\n\nRun the job on a cron schedule. No separate Celery Beat process — the scheduler runs inside your server.\n\n```arc\n@schedule \"0 9 * * *\"     // daily at 9am UTC\n@schedule \"*/15 * * * *\"  // every 15 minutes\n@schedule \"0 0 1 * *\"     // first of every month\njob CleanupOldSessions()\n  db.sessions.deleteMany({ expiresAt: { lt: now() } })\n```\n\nCron format: `min hour dom month dow` (5-field standard cron).\n\n### `@priority high|normal|low`\n\nControls dequeue order within a queue. High-priority jobs are processed before normal, normal before low.\n\n```arc\n@priority high\njob ProcessPayment(orderId: Int)\n  // dequeued before normal jobs\n```\n\n### `@retries \u003cn\u003e`\n\nOverride the default retry count (default: 3). Failed jobs retry with exponential backoff.\n\n```arc\n@retries 5\n@backoff 2000     // base backoff in ms (doubles each retry)\njob SyncInventory(productId: Int)\n  // retries up to 5 times: 2s, 4s, 8s, 16s, 32s\n```\n\n### `@timeout \u003cms\u003e`\n\nOverride the default job timeout (default: 30,000ms). Jobs that exceed this are treated as failures.\n\n```arc\n@timeout 300000   // 5-minute timeout\njob GenerateReport(reportId: Int)\n  // heavy computation\n```\n\n### `@concurrency \u003cn\u003e`\n\nLimit how many instances of this job run simultaneously across all workers.\n\n```arc\n@concurrency 2\njob ResizeImages(assetId: Int)\n  // max 2 running at a time\n```\n\n### `@unique`\n\n**celery-once equivalent.** Prevents duplicate jobs when one is already pending or running. Lock is keyed by job name + serialized args. Stale locks (crashed workers) auto-expire.\n\n```arc\n@unique                          // default: skip duplicates silently\n@unique strategy=skip            // same as above\n@unique strategy=reject          // throw JobAlreadyRunning error\n@unique strategy=replace         // cancel existing, enqueue new\n@unique timeout=3600000          // lock TTL in ms (default: 1 hour)\n@unique timeout=300000 strategy=skip\njob SendInvoice(invoiceId: Int)\n  email.send({ to: \"...\" })\n```\n\nCalling `SendInvoice(42)` twice while the first is pending → second is silently discarded (with `strategy=skip`).\n\n### `@progress`\n\nEnables `job.progress(pct)` inside the job body. Progress streams to `/_arc/jobs/:id/progress` via SSE, which can be consumed by `@live` pages.\n\n```arc\n@progress\njob ImportCSV(fileId: Int)\n  const rows = db.uploads.find(fileId)\n  for i, row in rows\n    job.progress((i + 1) / rows.length * 100, { processed: i + 1 })\n    processRow(row)\n```\n\n### `@then \u003cJobName\u003e`\n\nAuto-enqueue another job on success. Validated at compile time — `arc build` fails if `JobName` doesn't exist.\n\n```arc\n@then ProcessOrder\njob ValidateOrder(orderId: Int)\n  // ... validate; if this succeeds ProcessOrder(orderId) is auto-called\n\njob ProcessOrder(orderId: Int)\n  // ...\n```\n\n---\n\n## Calling Jobs\n\n```arc\n// Fire and forget (returns Promise\u003cstring\u003e job id)\nSendInvoice(invoiceId)\n\n// Delayed execution\nSendReminderEmail.delay(86400000, userId)   // delay in ms\n\n// Run at a specific time\nDailyDigest.at(new Date(\"2026-06-01T09:00:00Z\"))\n\n// Custom idempotency key\nProcessPayment.unique(\"order-42-payment\", orderId, amount)\n\n// Check job status from a route\n@route get \"/jobs/:id\" -\u003e Response\n  const status = await Queue.status(id)\n  json(status)\n```\n\n---\n\n## Queue Adapters\n\n### SQLite (default — zero ops)\n\nBest for: single-server apps, development, apps with \u003c ~10k jobs/min.\n\n```json\n{\n  \"queues\": {\n    \"default\": { \"backend\": \"sqlite\" }\n  }\n}\n```\n\n- **~15,000 jobs/sec** in WAL mode\n- Zero infrastructure — uses your existing `app.db`\n- Persistent across restarts\n- Jobs survive server crashes\n\n### Redis (high-throughput)\n\nBest for: high-volume apps, multi-worker deployments, horizontal scaling.\n\n```json\n{\n  \"queues\": {\n    \"default\": {\n      \"backend\": \"redis\",\n      \"url\": \"${REDIS_URL}\"\n    }\n  }\n}\n```\n\n- **100,000+ ops/sec**\n- Priority queues via sorted sets\n- `@unique` locks via atomic `SET NX PX`\n- Requires `Bun.Redis` (built-in) or `ioredis` (`bun add ioredis` for Node.js)\n\n### Mixed (recommended for production)\n\n```json\n{\n  \"queues\": {\n    \"default\":       { \"backend\": \"sqlite\" },\n    \"payments\":      { \"backend\": \"redis\", \"url\": \"${REDIS_URL}\" },\n    \"notifications\": { \"backend\": \"redis\", \"url\": \"${REDIS_URL}\" }\n  }\n}\n```\n\n---\n\n## Comparison\n\n### vs BullMQ \u0026 pg-boss\n\n| | arc-jobs (SQLite) | arc-jobs (Redis) | BullMQ | pg-boss |\n|---|---|---|---|---|\n| **Backend** | SQLite (in app.db) | Redis | Redis | PostgreSQL |\n| **Setup** | Zero | `bun add ioredis` | Redis required | Postgres required |\n| **Worker process** | In-process | In-process | Separate or in-process | In-process |\n| **Type safety** | Compile-time (Arc) | Compile-time (Arc) | Runtime | Runtime |\n| **Deduplication** | `@unique` built-in | `@unique` built-in | `jobId` option | `singletonKey` |\n| **Cron scheduler** | In-process | In-process | Separate or in-process | Built-in |\n| **Progress tracking** | `@progress` + SSE | `@progress` + SSE | `job.updateProgress()` | None |\n| **Admin dashboard** | `/_arc/jobs` built-in | `/_arc/jobs` built-in | Bull Board (extra pkg) | pgboss-web |\n| **Throughput** | ~15k jobs/min | ~100k+ ops/sec | ~100k+ ops/sec | ~5k jobs/sec |\n\n### vs Django Celery\n\n| | arc-jobs (SQLite) | arc-jobs (Redis) | Django Celery |\n|---|---|---|---|\n| **Enqueue latency** | ~0.1ms | ~0.5ms | ~1–5ms (network hop) |\n| **Throughput** | ~15k jobs/sec | ~100k jobs/sec | Millions/min (distributed) |\n| **Setup** | Zero | `bun add ioredis` | Redis + worker process + Celery Beat |\n| **Worker process** | In-process | In-process | Separate `celery worker` |\n| **Scheduler** | In-process | In-process | Separate `celery beat` |\n| **Type safety** | Compile-time | Compile-time | Runtime (stringly-typed) |\n| **`@unique`** | Built-in | Built-in | Requires celery-once |\n| **Test mode** | Synchronous flush | Synchronous flush | Needs mock broker |\n\nArc jobs on Bun outperform Python Celery for I/O-bound work (webhooks, email, API calls) — the common 80% case. Celery wins for CPU-bound tasks (ML, image processing) that benefit from multi-process parallelism.\n\n---\n\n## Arc Ecosystem\n\narc-jobs is a first-class part of the [Arc](https://arc-language.dev) web framework. Jobs are a built-in language construct — not a library bolted on.\n\n```\nArc ecosystem\n├── arc           compiler \u0026 language core    arc-language.dev\n├── arc-cms       admin panel + headless CMS  github.com/arc-language/arc-cms\n└── arc-jobs      background job queues       github.com/arc-language/arc-jobs\n```\n\n**How jobs fit into Arc's data contexts:**\n\n| Context | When to use |\n|---------|-------------|\n| `@state` | Instant client-side updates — no server needed |\n| `@live` | Edge-rendered HTML, one server round trip |\n| `@realtime` | WebSocket/SSE for live collaborative features |\n| **`job`** | **Background work: email, payments, imports, reports** |\n\nJobs are triggered from `@route` handlers and run asynchronously inside your server. Progress from `@progress` jobs streams to `@live` pages via SSE.\n\n---\n\n## Admin Dashboard\n\nA built-in dashboard is served at `/_arc/jobs` when your server is running:\n\n- **Overview**: pending / running / completed / failed counts per queue (live, 2s refresh)\n- **Active jobs**: elapsed time, progress bar for `@progress` jobs, cancel button\n- **Schedules**: cron expression + next fire time\n- **Active locks**: `@unique` locks with remaining TTL, force-unlock button\n- **Dead letter queue**: failed jobs with Replay button\n\n---\n\n## CLI\n\n```bash\n# Interactive queue setup\narc-jobs init\n\n# Show queue depths and job counts\narc-jobs stats\narc-jobs stats --db path/to/app.db\n\n# Replay dead letter queue\narc-jobs replay\narc-jobs replay --job SendInvoice         # specific job type\narc-jobs replay --db path/to/app.db\n\n# Real-time terminal monitor (refreshes every 2s)\narc-jobs monitor\narc-jobs monitor --db path/to/app.db\n\n# Version\narc-jobs --version\n```\n\n---\n\n## Testing\n\nIn `NODE_ENV=test`, all queues use a synchronous in-memory adapter that does **not** auto-process jobs. Call `Queue.flush()` explicitly to run them.\n\n```javascript\n// tests/jobs.test.js\nimport { Queue, SendInvoice, ProcessPayment } from './dist/server.test.js'\n\ntest('SendInvoice is enqueued on order creation', async () =\u003e {\n  Queue.reset()\n\n  const res = await fetch('http://localhost:3001/orders', {\n    method: 'POST',\n    body: JSON.stringify({ amount: 99 }),\n  })\n  assert.strictEqual(res.status, 201)\n  Queue.assertEnqueued('SendInvoice', [42])\n})\n\ntest('SendInvoice executes correctly', async () =\u003e {\n  Queue.reset()\n  await SendInvoice(42)\n  await Queue.flush()\n  assert.strictEqual(Queue.completed('SendInvoice').length, 1)\n  assert.strictEqual(Queue.dead().length, 0)\n})\n\n// Test @unique deduplication\ntest('duplicate SendInvoice calls are deduplicated', async () =\u003e {\n  Queue.reset()\n  await SendInvoice(42)\n  await SendInvoice(42)  // duplicate — skipped\n  assert.strictEqual(Queue.pending('SendInvoice').length, 1)\n})\n\n// Test @then chain\ntest('@then chain auto-enqueues ProcessOrder', async () =\u003e {\n  Queue.reset()\n  await ValidateOrder(99)\n  await Queue.flush()\n  assert.strictEqual(Queue.completed('ValidateOrder').length, 1)\n  await Queue.flush()  // process the auto-enqueued job\n  assert.strictEqual(Queue.completed('ProcessOrder').length, 1)\n})\n```\n\nBuild a test-mode server with `NODE_ENV=test arc build-server .`.\n\n---\n\n## Cloudflare Workers\n\nOn the Cloudflare target, arc-jobs maps to native CF primitives:\n\n| Feature | CF Primitive |\n|---|---|\n| Queue | CF Queues binding |\n| `@schedule` | CF Cron Triggers (in `wrangler.toml`) |\n| `@unique` | Durable Objects |\n| `@progress` | Durable Objects state |\n\nNo configuration needed — `arc build --target cloudflare` handles it automatically.\n\n---\n\n## Contributing\n\nContributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, branch naming, and the PR checklist.\n\nTo report a security vulnerability privately, see [SECURITY.md](SECURITY.md).\n\n---\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n\nPart of the [Arc](https://arc-language.dev) ecosystem.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farc-language%2Farc-jobs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farc-language%2Farc-jobs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farc-language%2Farc-jobs/lists"}