{"id":30029089,"url":"https://github.com/noxify/vorsteh-queue","last_synced_at":"2026-04-17T08:02:10.536Z","repository":{"id":305041926,"uuid":"1020839839","full_name":"noxify/vorsteh-queue","owner":"noxify","description":"A powerful, ORM-agnostic queue engine for PostgreSQL 12+. Handle background jobs, scheduled tasks, and recurring processes with ease.","archived":false,"fork":false,"pushed_at":"2025-07-28T17:41:58.000Z","size":4221,"stargazers_count":1,"open_issues_count":5,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-28T19:14:07.694Z","etag":null,"topics":["adapters","agnostic","background-jobs","cron","delayed-jobs","drizzle","job-queue","mariadb","orm","postgres","postgressql","prisma","queue","queueing","scheduler","task-queue","timezone","turbo","typescript","worker"],"latest_commit_sha":null,"homepage":"https://vorsteh-queue.dev","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/noxify.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-16T13:25:11.000Z","updated_at":"2025-07-22T20:56:27.000Z","dependencies_parsed_at":"2025-07-18T03:17:38.080Z","dependency_job_id":"dc10ac89-3787-490a-8845-160fbaaced41","html_url":"https://github.com/noxify/vorsteh-queue","commit_stats":null,"previous_names":["noxify/vorsteh-queue"],"tags_count":13,"template":false,"template_full_name":null,"purl":"pkg:github/noxify/vorsteh-queue","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noxify%2Fvorsteh-queue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noxify%2Fvorsteh-queue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noxify%2Fvorsteh-queue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noxify%2Fvorsteh-queue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/noxify","download_url":"https://codeload.github.com/noxify/vorsteh-queue/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noxify%2Fvorsteh-queue/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269118261,"owners_count":24362990,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-06T02:00:09.910Z","response_time":99,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adapters","agnostic","background-jobs","cron","delayed-jobs","drizzle","job-queue","mariadb","orm","postgres","postgressql","prisma","queue","queueing","scheduler","task-queue","timezone","turbo","typescript","worker"],"created_at":"2025-08-06T17:01:29.793Z","updated_at":"2026-04-17T08:02:10.522Z","avatar_url":"https://github.com/noxify.png","language":"TypeScript","readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"./assets/vorsteh-queue-logo-nobg.png\" alt=\"Vorsteh Queue\" height=\"200\" /\u003e\n  \u003ch1\u003eVorsteh Queue\u003c/h1\u003e\n  \u003cp\u003eA powerful, ORM-agnostic queue engine for PostgreSQL 12+. Handle background jobs, scheduled tasks, and recurring processes with ease.\u003c/p\u003e\n\u003c/div\u003e\n\n## Features\n\n- **Type-safe**: Full TypeScript support with generic job payloads\n- **Multiple adapters**: Drizzle ORM (PostgreSQL), Prisma ORM (PostgreSQL), Kysely (PostgreSQL) and in-memory implementations\n- **Priority queues**: Numeric priority system (lower = higher priority)\n- **Delayed jobs**: Schedule jobs for future execution\n- **Recurring jobs**: Cron expressions and interval-based repetition\n- **Batch processing**: Process multiple jobs concurrently for higher efficiency and better performance by supporting parallel execution and grouping of jobs.\n- **UTC-first timezone support**: Reliable timezone handling with UTC storage\n- **Progress tracking**: Real-time job progress updates\n- **Event system**: Listen to job lifecycle events\n- **Graceful shutdown**: Clean job processing termination\n\n## Repository Structure\n\n```\n.\n├── packages/\n│   ├── core/                # Core queue logic and interfaces\n│   ├── adapter-drizzle/     # Drizzle ORM adapter (PostgreSQL)\n│   └── adapter-prisma/      # Prisma ORM adapter (PostgreSQL)\n│   └── adapter-kysely/      # Kysely adapter (PostgreSQL)\n├── examples/                # Standalone usage examples\n└── tooling/                # Shared development tools\n```\n\n## Quick Start\n\n### Requirements\n\n- **Node.js 20+**\n- **ESM only** - This package is ESM-only and cannot be imported with `require()`\n\n### Installation\n\n```bash\nnpm install @vorsteh-queue/core @vorsteh-queue/adapter-drizzle\n# or\npnpm add @vorsteh-queue/core @vorsteh-queue/adapter-drizzle\n```\n\n\u003e **Note**: Make sure your project has `\"type\": \"module\"` in package.json or use `.mjs` file extensions.\n\n### Basic Usage\n\n```typescript\n// Drizzle ORM with PostgreSQL\n\n// Prisma ORM with PostgreSQL\nimport { PrismaClient } from \"@prisma/client\"\nimport { drizzle } from \"drizzle-orm/node-postgres\"\nimport { Pool } from \"pg\"\n\nimport { PostgresQueueAdapter } from \"@vorsteh-queue/adapter-drizzle\"\nimport { PostgresPrismaQueueAdapter } from \"@vorsteh-queue/adapter-prisma\"\nimport { Queue } from \"@vorsteh-queue/core\"\n\ninterface EmailPayload {\n  to: string\n  subject: string\n  body: string\n}\n\ninterface EmailResult {\n  messageId: string\n  sent: boolean\n}\n\nconst pool = new Pool({ connectionString: \"postgresql://...\" })\nconst db = drizzle(pool)\nconst queue = new Queue(new PostgresQueueAdapter(db), { name: \"my-queue\" })\n\nconst prisma = new PrismaClient()\nconst queue = new Queue(new PostgresPrismaQueueAdapter(prisma), { name: \"my-queue\" })\n\n// Register job handlers\nqueue.register\u003cEmailPayload, EmailResult\u003e(\"send-email\", async (job) =\u003e {\n  console.log(`Sending email to ${job.payload.to}`)\n\n  // Send email logic here\n  await sendEmail(job.payload)\n\n  // Return result - will be stored in job.result field\n  return {\n    messageId: \"msg_123\",\n    sent: true,\n  }\n})\n\n// Add jobs\nawait queue.add(\"send-email\", {\n  to: \"user@example.com\",\n  subject: \"Welcome!\",\n  body: \"Welcome to our service!\",\n})\nawait queue.add(\n  \"send-email\",\n  { to: \"admin@example.com\", subject: \"Report\" },\n  {\n    priority: 1, // Higher priority\n    delay: 5000, // Delay 5 seconds\n  },\n)\n\n// Start processing\nqueue.start()\n```\n\n## Examples\n\nCheck out the [examples directory](./examples/) for complete, runnable examples.\n\n\u003e **Note**: All examples demonstrate the UTC-first timezone approach and automatic job cleanup features.\n\n## Priority System\n\nJobs are processed by priority (lower number = higher priority):\n\n```typescript\nawait queue.add(\"urgent-task\", payload, { priority: 1 }) // Processed first\nawait queue.add(\"normal-task\", payload, { priority: 2 }) // Default priority\nawait queue.add(\"low-task\", payload, { priority: 3 }) // Processed last\n```\n\n## Recurring Jobs\n\n```typescript\n// Cron expression\nawait queue.add(\"daily-report\", payload, {\n  cron: \"0 9 * * *\", // Every day at 9 AM\n})\n\n// Interval with limit\nawait queue.add(\"health-check\", payload, {\n  repeat: { every: 30000, limit: 10 }, // Every 30s, 10 times\n})\n```\n\n## Batch Processing\n\nProcess multiple jobs in a single batch for higher throughput and efficiency.\n\n```typescript\nqueue.registerBatch\u003c{ file: string }, { ok: boolean }\u003e(\"process-files\", async (jobs) =\u003e {\n  console.log(`Processing batch of ${jobs.length} files...`)\n  return jobs.map(() =\u003e ({ ok: true }))\n})\n\nawait queue.addJobs(\"process-files\", [{ file: \"a.csv\" }, { file: \"b.csv\" }, { file: \"c.csv\" }])\n\nqueue.on(\"batch:processing\", (jobs) =\u003e {\n  console.log(`Batch started: ${jobs.length} jobs`)\n})\nqueue.on(\"batch:completed\", (jobs) =\u003e {\n  console.log(`Batch completed: ${jobs.length} jobs`)\n})\n```\n\n## Job Cleanup\n\n```typescript\n// Automatic cleanup configuration\nconst queue = new Queue(adapter, {\n  name: \"my-queue\",\n  removeOnComplete: true, // Remove completed jobs immediately\n  removeOnFail: false, // Keep failed jobs for debugging\n  // Or use numbers to keep N jobs\n  removeOnComplete: 100, // Keep last 100 completed jobs\n  removeOnFail: 50, // Keep last 50 failed jobs\n})\n```\n\n## Timezone Support\n\nVorsteh Queue uses a **UTC-first approach** for reliable timezone handling:\n\n```typescript\n// Schedule job for 9 AM New York time - converted to UTC immediately\nawait queue.add(\"daily-report\", payload, {\n  cron: \"0 9 * * *\",\n  timezone: \"America/New_York\", // Timezone used for conversion only\n})\n\n// Schedule job for specific time in Tokyo - stored as UTC\nawait queue.add(\"notification\", payload, {\n  runAt: new Date(\"2024-01-15T10:00:00\"),\n  timezone: \"Asia/Tokyo\", // Interprets runAt in Tokyo time\n})\n\n// Complex cron with timezone - result always UTC\nawait queue.add(\"business-task\", payload, {\n  cron: \"*/15 9-17 * * 1-5\",\n  timezone: \"Europe/London\", // Business hours in London time\n})\n```\n\n### How It Works\n\n1. **Timezone conversion happens at job creation** - not at execution\n2. **All timestamps stored in database are UTC** - no timezone ambiguity\n3. **Recurring jobs recalculate using original timezone** - maintains accuracy\n4. **Simple and predictable** - no runtime timezone complexity\n5. **Server timezone independent** - works consistently across environments\n\n## Job Results\n\nJob handlers can return results that are automatically stored and made available:\n\n```typescript\ninterface ProcessResult {\n  processed: number\n  errors: string[]\n  duration: number\n}\n\nqueue.register\u003c{ items: string[] }, ProcessResult\u003e(\"process-data\", async (job) =\u003e {\n  const startTime = Date.now()\n  const errors: string[] = []\n  let processed = 0\n\n  for (const item of job.payload.items) {\n    try {\n      await processItem(item)\n      processed++\n    } catch (error) {\n      errors.push(`Failed to process ${item}: ${error.message}`)\n    }\n  }\n\n  // Return result - automatically stored in job.result field\n  return {\n    processed,\n    errors,\n    duration: Date.now() - startTime,\n  }\n})\n\n// Access results in events\nqueue.on(\"job:completed\", (job) =\u003e {\n  const result = job.result as ProcessResult\n  console.log(`Processed ${result.processed} items in ${result.duration}ms`)\n  if (result.errors.length \u003e 0) {\n    console.warn(`Errors: ${result.errors.join(\", \")}`)\n  }\n})\n```\n\n## Progress Tracking\n\n```typescript\n// Register a job that reports progress\nqueue.register(\"process-data\", async (job) =\u003e {\n  const items = job.payload.items\n\n  for (let i = 0; i \u003c items.length; i++) {\n    // Process item\n    await processItem(items[i])\n\n    // Update progress (0-100)\n    const progress = Math.round(((i + 1) / items.length) * 100)\n    await job.updateProgress(progress)\n  }\n\n  return { processed: items.length }\n})\n\n// Listen to progress updates\nqueue.on(\"job:progress\", (job) =\u003e {\n  console.log(`Job ${job.name}: ${job.progress}% complete`)\n})\n```\n\n## Event System\n\n```typescript\n// Listen to all job lifecycle events\nqueue.on(\"job:added\", (job) =\u003e {\n  console.log(`✅ Job ${job.name} added to queue`)\n})\n\nqueue.on(\"job:processing\", (job) =\u003e {\n  console.log(`⚡ Job ${job.name} started processing`)\n})\n\nqueue.on(\"job:completed\", (job) =\u003e {\n  console.log(`🎉 Job ${job.name} completed successfully`)\n  console.log(`📊 Result:`, job.result) // Access job result\n})\n\nqueue.on(\"job:failed\", (job) =\u003e {\n  console.error(`❌ Job ${job.name} failed: ${job.error}`)\n})\n\nqueue.on(\"job:retried\", (job) =\u003e {\n  console.log(`🔄 Job ${job.name} retrying (attempt ${job.attempts})`)\n})\n\n// Queue-level events\nqueue.on(\"queue:paused\", () =\u003e {\n  console.log(\"⏸️ Queue paused\")\n})\n\nqueue.on(\"queue:resumed\", () =\u003e {\n  console.log(\"▶️ Queue resumed\")\n})\n```\n\n## Architecture\n\n### UTC-First Design\n\n- **Database storage**: All timestamps stored as UTC\n- **Timezone conversion**: Happens at job creation time\n- **No timezone fields**: Database schema contains no timezone columns\n- **Predictable behavior**: Same results across different server timezones\n\n### Adapter Pattern\n\n- **Pluggable storage**: Easy to add new database adapters\n- **Consistent interface**: Same API across all adapters\n- **Type safety**: Full TypeScript support throughout\n\n## Development\n\nThis project was developed with AI assistance, combining human expertise with AI-powered code generation to accelerate development while maintaining high code quality standards.\n\n```bash\n# Install dependencies\npnpm install\n\n# Autofix format issues\npnpm format:fix\n\n# Fix issues in the `package.json` ( order, version mismatch )\npnpm lint:ws -f\n\n# Lint\npnpm lint\n\n# Type check\npnpm typecheck\n\n# Run all tests\npnpm test\n\n# Build packages\npnpm build\n```\n\n## Contributing\n\nContributions are welcome! Please read our [contributing guidelines](./CONTRIBUTING.md) and submit pull requests to our repository.\n\n## License\n\nMIT License - see [LICENSE](./LICENSE) file for details.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnoxify%2Fvorsteh-queue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnoxify%2Fvorsteh-queue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnoxify%2Fvorsteh-queue/lists"}