{"id":35357542,"url":"https://github.com/boringnode/queue","last_synced_at":"2026-03-03T21:04:06.964Z","repository":{"id":328405363,"uuid":"857318095","full_name":"boringnode/queue","owner":"boringnode","description":"A simple and efficient framework-agnostic queue system for Node.js applications","archived":false,"fork":false,"pushed_at":"2026-01-26T19:01:05.000Z","size":383,"stargazers_count":87,"open_issues_count":1,"forks_count":1,"subscribers_count":5,"default_branch":"main","last_synced_at":"2026-01-27T06:37:29.435Z","etag":null,"topics":["framework-agnostic","nodejs","queue","queue-workers"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/boringnode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/funding.yml","license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":["RomainLanz"]}},"created_at":"2024-09-14T10:35:22.000Z","updated_at":"2026-01-26T22:43:24.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/boringnode/queue","commit_stats":null,"previous_names":["boringnode/queue"],"tags_count":13,"template":false,"template_full_name":null,"purl":"pkg:github/boringnode/queue","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boringnode%2Fqueue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boringnode%2Fqueue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boringnode%2Fqueue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boringnode%2Fqueue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/boringnode","download_url":"https://codeload.github.com/boringnode/queue/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boringnode%2Fqueue/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28810475,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-27T07:41:26.337Z","status":"ssl_error","status_checked_at":"2026-01-27T07:41:08.776Z","response_time":168,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["framework-agnostic","nodejs","queue","queue-workers"],"created_at":"2026-01-01T23:30:12.924Z","updated_at":"2026-01-27T09:18:34.834Z","avatar_url":"https://github.com/boringnode.png","language":"TypeScript","readme":"# @boringnode/queue\n\n\u003cdiv align=\"center\"\u003e\n\n[![typescript-image]][typescript-url]\n[![gh-workflow-image]][gh-workflow-url]\n[![npm-image]][npm-url]\n[![npm-download-image]][npm-download-url]\n[![license-image]][license-url]\n\n\u003c/div\u003e\n\nA simple and efficient queue system for Node.js applications. Built for simplicity and ease of use, `@boringnode/queue` allows you to dispatch background jobs and process them asynchronously with support for multiple queue adapters.\n\n## Installation\n\n```bash\nnpm install @boringnode/queue\n```\n\n## Features\n\n- **Multiple Queue Adapters**: Redis, Knex (PostgreSQL, MySQL, SQLite), and Sync\n- **Type-Safe Jobs**: TypeScript classes with typed payloads\n- **Delayed Jobs**: Schedule jobs to run after a delay\n- **Priority Queues**: Process high-priority jobs first\n- **Bulk Dispatch**: Efficiently dispatch thousands of jobs at once\n- **Job Grouping**: Organize related jobs for monitoring\n- **Retry with Backoff**: Exponential, linear, or fixed backoff strategies\n- **Job Timeout**: Fail or retry jobs that exceed a time limit\n- **Job History**: Retain completed/failed jobs for debugging\n- **Scheduled Jobs**: Cron or interval-based recurring jobs\n- **Auto-Discovery**: Automatically register jobs from specified locations\n\n## Quick Start\n\n### 1. Define a Job\n\n```typescript\nimport { Job } from '@boringnode/queue'\nimport type { JobOptions } from '@boringnode/queue/types'\n\ninterface SendEmailPayload {\n  to: string\n}\n\nexport default class SendEmailJob extends Job\u003cSendEmailPayload\u003e {\n  static options: JobOptions = {\n    queue: 'email',\n  }\n\n  async execute(): Promise\u003cvoid\u003e {\n    console.log(`Sending email to: ${this.payload.to}`)\n  }\n}\n```\n\n\u003e [!NOTE]\n\u003e The job name defaults to the class name (`SendEmailJob`). You can override it with `name: 'CustomName'` in options.\n\n\u003e [!WARNING]\n\u003e If you minify your code in production, class names may be mangled. Always specify `name` explicitly in your job options.\n\n### 2. Configure the Queue Manager\n\n```typescript\nimport { QueueManager } from '@boringnode/queue'\nimport { redis } from '@boringnode/queue/drivers/redis_adapter'\n\nawait QueueManager.init({\n  default: 'redis',\n  adapters: {\n    redis: redis({ host: 'localhost', port: 6379 }),\n  },\n  locations: ['./app/jobs/**/*.ts'],\n})\n```\n\n### 3. Dispatch Jobs\n\n```typescript\n// Simple dispatch\nawait SendEmailJob.dispatch({ to: 'user@example.com' })\n\n// With options\nawait SendEmailJob.dispatch({ to: 'user@example.com' })\n  .toQueue('high-priority')\n  .priority(1)\n  .in('5m')\n```\n\n### 4. Start a Worker\n\n```typescript\nimport { Worker } from '@boringnode/queue'\n\nconst worker = new Worker(config)\nawait worker.start(['default', 'email'])\n```\n\n## Bulk Dispatch\n\nEfficiently dispatch thousands of jobs in a single batch operation:\n\n```typescript\nconst { jobIds } = await SendEmailJob.dispatchMany([\n  { to: 'user1@example.com' },\n  { to: 'user2@example.com' },\n  { to: 'user3@example.com' },\n])\n  .group('newsletter-jan-2025')\n  .toQueue('emails')\n  .priority(3)\n\nconsole.log(`Dispatched ${jobIds.length} jobs`)\n```\n\nThis uses Redis MULTI/EXEC or SQL batch insert for optimal performance.\n\n## Job Grouping\n\nOrganize related jobs together for monitoring and filtering:\n\n```typescript\n// Group newsletter jobs\nawait SendEmailJob.dispatch({ to: 'user@example.com' }).group('newsletter-jan-2025')\n\n// Group with bulk dispatch\nawait SendEmailJob.dispatchMany(recipients).group('newsletter-jan-2025')\n```\n\nThe `groupId` is stored with job data and accessible via `job.data.groupId`.\n\n## Job History \u0026 Retention\n\nKeep completed and failed jobs for debugging:\n\n```typescript\nexport default class ImportantJob extends Job\u003cPayload\u003e {\n  static options: JobOptions = {\n    // Keep last 1000 completed jobs\n    removeOnComplete: { count: 1000 },\n\n    // Keep failed jobs for 7 days\n    removeOnFail: { age: '7d' },\n  }\n}\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eRetention options\u003c/strong\u003e\u003c/summary\u003e\n\n| Value                       | Behavior           |\n| --------------------------- | ------------------ |\n| `true` (default)            | Remove immediately |\n| `false`                     | Keep forever       |\n| `{ count: n }`              | Keep last n jobs   |\n| `{ age: '7d' }`             | Keep for duration  |\n| `{ count: 100, age: '1d' }` | Both limits apply  |\n\nQuery job history:\n\n```typescript\nconst job = await adapter.getJob('job-id', 'queue-name')\nconsole.log(job.status) // 'completed' | 'failed'\nconsole.log(job.finishedAt) // timestamp\nconsole.log(job.error) // error message (if failed)\n```\n\n\u003c/details\u003e\n\n## Adapters\n\n### Redis (recommended for production)\n\n```typescript\nimport { redis } from '@boringnode/queue/drivers/redis_adapter'\n\n// With options\nconst adapter = redis({ host: 'localhost', port: 6379 })\n\n// With existing ioredis instance\nimport { Redis } from 'ioredis'\nconst connection = new Redis({ host: 'localhost' })\nconst adapter = redis(connection)\n```\n\n### Knex (PostgreSQL, MySQL, SQLite)\n\n```typescript\nimport { knex } from '@boringnode/queue/drivers/knex_adapter'\n\nconst adapter = knex({\n  client: 'pg',\n  connection: { host: 'localhost', database: 'myapp' },\n})\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eMore Knex examples\u003c/strong\u003e\u003c/summary\u003e\n\n```typescript\n// With existing Knex instance\nimport Knex from 'knex'\nconst connection = Knex({ client: 'pg', connection: '...' })\nconst adapter = knex(connection)\n\n// Custom table name\nconst adapter = knex(config, 'custom_jobs_table')\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eDatabase setup with QueueSchemaService\u003c/strong\u003e\u003c/summary\u003e\n\nThe Knex adapter requires tables to be created before use. Use `QueueSchemaService` to create them:\n\n```typescript\nimport { QueueSchemaService } from '@boringnode/queue'\nimport Knex from 'knex'\n\nconst connection = Knex({ client: 'pg', connection: '...' })\nconst schemaService = new QueueSchemaService(connection)\n\n// Create tables with default names\nawait schemaService.createJobsTable()\nawait schemaService.createSchedulesTable()\n\n// Or extend with custom columns\nawait schemaService.createJobsTable('queue_jobs', (table) =\u003e {\n  table.string('tenant_id', 255).nullable()\n})\n```\n\n**AdonisJS migration example:**\n\n```typescript\nimport { BaseSchema } from '@adonisjs/lucid/schema'\nimport { QueueSchemaService } from '@boringnode/queue'\n\nexport default class extends BaseSchema {\n  async up() {\n    const schemaService = new QueueSchemaService(this.db.connection().getWriteClient())\n    await schemaService.createJobsTable()\n    await schemaService.createSchedulesTable()\n  }\n\n  async down() {\n    const schemaService = new QueueSchemaService(this.db.connection().getWriteClient())\n    await schemaService.dropSchedulesTable()\n    await schemaService.dropJobsTable()\n  }\n}\n```\n\n\u003c/details\u003e\n\n### Fake (testing + assertions)\n\n```typescript\nimport { QueueManager } from '@boringnode/queue'\nimport { redis } from '@boringnode/queue/drivers/redis_adapter'\n\nawait QueueManager.init({\n  default: 'redis',\n  adapters: {\n    redis: redis({ host: 'localhost' }),\n  },\n  locations: ['./app/jobs/**/*.ts'],\n})\n\nconst adapter = QueueManager.fake()\n\nawait SendEmailJob.dispatch({ to: 'user@example.com' })\n\nadapter.assertPushed(SendEmailJob)\nadapter.assertPushed(SendEmailJob, {\n  queue: 'default',\n  payload: (payload) =\u003e payload.to === 'user@example.com',\n})\nadapter.assertPushedCount(1)\n\nQueueManager.restore()\n```\n\n### Sync (for testing)\n\n```typescript\nimport { sync } from '@boringnode/queue/drivers/sync_adapter'\n\nconst adapter = sync() // Jobs execute immediately\n```\n\n## Job Options\n\n```typescript\nexport default class MyJob extends Job\u003cPayload\u003e {\n  static options: JobOptions = {\n    queue: 'email', // Queue name (default: 'default')\n    priority: 1, // Lower = higher priority (default: 5)\n    maxRetries: 3, // Retry attempts before failing\n    timeout: '30s', // Max execution time\n    failOnTimeout: true, // Fail permanently on timeout (default: retry)\n    removeOnComplete: { count: 100 }, // Keep last 100 completed\n    removeOnFail: { age: '7d' }, // Keep failed for 7 days\n  }\n}\n```\n\n## Delayed Jobs\n\n```typescript\nawait SendEmailJob.dispatch(payload).in('30s') // 30 seconds\nawait SendEmailJob.dispatch(payload).in('5m') // 5 minutes\nawait SendEmailJob.dispatch(payload).in('2h') // 2 hours\nawait SendEmailJob.dispatch(payload).in('1d') // 1 day\n```\n\n## Retry \u0026 Backoff\n\n```typescript\nimport { exponentialBackoff } from '@boringnode/queue'\n\nexport default class ReliableJob extends Job\u003cPayload\u003e {\n  static options: JobOptions = {\n    maxRetries: 5,\n    retry: {\n      backoff: () =\u003e\n        exponentialBackoff({\n          baseDelay: '1s',\n          maxDelay: '1m',\n          multiplier: 2,\n          jitter: true,\n        }),\n    },\n  }\n}\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eAvailable strategies\u003c/strong\u003e\u003c/summary\u003e\n\n```typescript\nimport { exponentialBackoff, linearBackoff, fixedBackoff } from '@boringnode/queue'\n\n// Exponential: 1s, 2s, 4s, 8s...\nexponentialBackoff({ baseDelay: '1s', maxDelay: '1m', multiplier: 2 })\n\n// Linear: 1s, 2s, 3s, 4s...\nlinearBackoff({ baseDelay: '1s', maxDelay: '30s', multiplier: 1 })\n\n// Fixed: 5s, 5s, 5s...\nfixedBackoff({ baseDelay: '5s', jitter: true })\n```\n\n\u003c/details\u003e\n\n## Job Timeout\n\n```typescript\nexport default class LongRunningJob extends Job\u003cPayload\u003e {\n  static options: JobOptions = {\n    timeout: '30s',\n    failOnTimeout: false, // Will retry (default)\n  }\n\n  async execute(): Promise\u003cvoid\u003e {\n    for (const item of this.payload.items) {\n      // Check abort signal for graceful timeout handling\n      if (this.signal?.aborted) {\n        throw new Error('Job timed out')\n      }\n      await this.processItem(item)\n    }\n  }\n}\n```\n\n## Job Context\n\nAccess execution metadata via `this.context`:\n\n```typescript\nasync execute(): Promise\u003cvoid\u003e {\n  console.log(this.context.jobId)       // Unique job ID\n  console.log(this.context.attempt)     // 1, 2, 3...\n  console.log(this.context.queue)       // Queue name\n  console.log(this.context.priority)    // Priority value\n  console.log(this.context.acquiredAt)  // When acquired\n  console.log(this.context.stalledCount) // Stall recoveries\n}\n```\n\n## Scheduled Jobs\n\nRun jobs on a recurring basis:\n\n```typescript\n// Every 10 seconds\nawait MetricsJob.schedule({ endpoint: '/health' }).every('10s')\n\n// Cron schedule\nawait CleanupJob.schedule({ days: 30 })\n  .id('daily-cleanup')\n  .cron('0 0 * * *') // Midnight daily\n  .timezone('Europe/Paris')\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eSchedule management\u003c/strong\u003e\u003c/summary\u003e\n\n```typescript\nimport { Schedule } from '@boringnode/queue'\n\n// Find and manage\nconst schedule = await Schedule.find('daily-cleanup')\nawait schedule.pause()\nawait schedule.resume()\nawait schedule.trigger() // Run now\nawait schedule.delete()\n\n// List schedules\nconst all = await Schedule.list()\nconst active = await Schedule.list({ status: 'active' })\n```\n\n**Schedule options:**\n\n| Method              | Description                       |\n| ------------------- | --------------------------------- |\n| `.id(string)`       | Unique identifier                 |\n| `.every(duration)`  | Fixed interval ('5s', '1m', '1h') |\n| `.cron(expression)` | Cron schedule                     |\n| `.timezone(tz)`     | Timezone (default: 'UTC')         |\n| `.from(date)`       | Start boundary                    |\n| `.to(date)`         | End boundary                      |\n| `.limit(n)`         | Maximum runs                      |\n\n\u003c/details\u003e\n\n## Dependency Injection\n\nIntegrate with IoC containers:\n\n```typescript\nawait QueueManager.init({\n  // ...\n  jobFactory: async (JobClass) =\u003e {\n    return app.container.make(JobClass)\n  },\n})\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eExample with injected services\u003c/strong\u003e\u003c/summary\u003e\n\n```typescript\nexport default class SendEmailJob extends Job\u003cSendEmailPayload\u003e {\n  constructor(\n    private mailer: MailerService,\n    private logger: Logger\n  ) {\n    super()\n  }\n\n  async execute(): Promise\u003cvoid\u003e {\n    this.logger.info(`Sending email to ${this.payload.to}`)\n    await this.mailer.send(this.payload)\n  }\n}\n```\n\n\u003c/details\u003e\n\n## Worker Configuration\n\n```typescript\nconst config = {\n  worker: {\n    concurrency: 5, // Parallel jobs\n    idleDelay: '2s', // Poll interval when idle\n    timeout: '1m', // Default job timeout\n    stalledThreshold: '30s', // When to consider job stalled\n    stalledInterval: '30s', // How often to check\n    maxStalledCount: 1, // Max recoveries before failing\n    gracefulShutdown: true, // Wait for jobs on SIGTERM\n  },\n}\n```\n\n## Logging\n\n```typescript\nimport { pino } from 'pino'\n\nawait QueueManager.init({\n  // ...\n  logger: pino(),\n})\n```\n\n## Benchmarks\n\nPerformance comparison with BullMQ (5ms simulated work per job):\n\n| Jobs | Concurrency | @boringnode/queue | BullMQ | Diff        |\n| ---- | ----------- | ----------------- | ------ | ----------- |\n| 1000 | 5           | 1096ms            | 1116ms | 1.8% faster |\n| 1000 | 10          | 565ms             | 579ms  | 2.4% faster |\n| 100K | 10          | 56.2s             | 57.5s  | 2.1% faster |\n| 100K | 20          | 29.1s             | 29.6s  | 1.7% faster |\n\n```bash\nnpm run benchmark -- --realistic\n```\n\n[gh-workflow-image]: https://img.shields.io/github/actions/workflow/status/boringnode/queue/checks.yml?branch=main\u0026style=for-the-badge\n[gh-workflow-url]: https://github.com/boringnode/queue/actions/workflows/checks.yml\n[npm-image]: https://img.shields.io/npm/v/@boringnode/queue.svg?style=for-the-badge\u0026logo=npm\n[npm-url]: https://www.npmjs.com/package/@boringnode/queue\n[npm-download-image]: https://img.shields.io/npm/dm/@boringnode/queue?style=for-the-badge\n[npm-download-url]: https://www.npmjs.com/package/@boringnode/queue\n[typescript-image]: https://img.shields.io/badge/Typescript-294E80.svg?style=for-the-badge\u0026logo=typescript\n[typescript-url]: https://www.typescriptlang.org\n[license-image]: https://img.shields.io/npm/l/@boringnode/queue?color=blueviolet\u0026style=for-the-badge\n[license-url]: LICENSE.md\n","funding_links":["https://github.com/sponsors/RomainLanz"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fboringnode%2Fqueue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fboringnode%2Fqueue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fboringnode%2Fqueue/lists"}