{"id":16695910,"url":"https://github.com/justplainstuff/plainjob","last_synced_at":"2025-04-10T02:14:17.384Z","repository":{"id":255569536,"uuid":"852047278","full_name":"justplainstuff/plainjob","owner":"justplainstuff","description":"SQLite-backed job queue processing 15k jobs/s ","archived":false,"fork":false,"pushed_at":"2025-03-13T23:04:27.000Z","size":183,"stargazers_count":38,"open_issues_count":5,"forks_count":2,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-04-10T02:14:12.189Z","etag":null,"topics":["better-sqlite3","bun","job-queue","jobs","queue","sqlite","task-queue","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/justplainstuff.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-04T06:12:59.000Z","updated_at":"2025-03-26T16:04:36.000Z","dependencies_parsed_at":"2024-09-17T13:23:57.613Z","dependency_job_id":"26a1107c-e14a-4009-a92a-67a46be67e00","html_url":"https://github.com/justplainstuff/plainjob","commit_stats":null,"previous_names":["justplainstuff/plainjobs","justplainstuff/plainjob"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justplainstuff%2Fplainjob","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justplainstuff%2Fplainjob/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justplainstuff%2Fplainjob/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justplainstuff%2Fplainjob/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/justplainstuff","download_url":"https://codeload.github.com/justplainstuff/plainjob/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248142902,"owners_count":21054671,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["better-sqlite3","bun","job-queue","jobs","queue","sqlite","task-queue","typescript"],"created_at":"2024-10-12T17:24:11.412Z","updated_at":"2025-04-10T02:14:17.358Z","avatar_url":"https://github.com/justplainstuff.png","language":"TypeScript","readme":"# plainjob\n\nA SQLite-backed job queue for `better-sqlite3` and `bun:sqlite` processing 15k jobs/s.\n\n## Getting Started\n\n### Using bun\n\n```bash\nbun add plainjob\n```\n\nCreate a queue:\n\n```typescript\nimport { bun, defineQueue } from \"plainjob\";\nimport Database from \"bun:sqlite\";\n\nconst connection = bun(new Database(\"data.db\", { strict: true }));\nconst queue = defineQueue({ connection });\n```\n\nMake sure strict mode is enabled!\n\n### Using node\n\n```bash\nnpm install plainjob better-sqlite3\n```\n\nCreate a queue:\n\n```typescript\nimport { better, defineQueue } from \"plainjob\";\nimport Database from \"better-sqlite3\";\n\nconst connection = better(new Database(\"data.db\"));\nconst queue = defineQueue({ connection });\n```\n\n### Minimal Example\n\n```typescript\nimport { bun, defineQueue, defineWorker } from \"plainjob\";\nimport Database from \"bun:sqlite\";\n\nconst connection = bun(new Database(\"data.db\", { strict: true }));\nconst queue = defineQueue({ connection });\n\n// Define a worker\nconst worker = defineWorker(\n  \"print\",\n  async (job) =\u003e {\n    console.log(`Processing job ${job.id}: ${job.data}`);\n  },\n  { queue }\n);\n\n// Add a job\nqueue.add(\"print\", \"Hello, plainjob!\");\n\n// Start the worker\nworker.start();\n```\n\n## Features\n\n- **SQLite-backed**: Reliable persistence using bun:sqlite or better-sqlite3\n- **High performance**: Process up to 15,000 jobs per second\n- **Cron-scheduled jobs**: Easily schedule recurring tasks\n- **Delayed jobs**: Run jobs after a specified delay\n- **Automatic job cleanup**: Remove old completed and failed jobs\n- **Job timeout handling**: Re-queue jobs if a worker dies\n- **Custom logging**: Integrate with your preferred logging solution\n- **Lightweight**: No external dependencies beyond better-sqlite3 and a cron-parser\n\n## Usage\n\n### Creating a Queue\n\n```typescript\nimport { bun, defineQueue } from \"plainjob\";\nimport Database from \"bun:sqlite\";\n\nconst connection = bun(new Database(\"data.db\", { strict: true }));\nconst queue = defineQueue({\n  connection,\n  timeout: 30 * 60 * 1000, // 30 minutes\n  removeDoneJobsOlderThan: 7 * 24 * 60 * 60 * 1000, // 7 days\n  removeFailedJobsOlderThan: 30 * 24 * 60 * 60 * 1000, // 30 days\n});\n```\n\n### Adding Jobs\n\n```typescript\n// Enqueue a one-time job\nqueue.add(\"send-email\", { to: \"user@example.com\", subject: \"Hello\" });\n\n// Run a job a after 1 second\nqueue.add(\n  \"send-email\",\n  { to: \"user@example.com\", subject: \"Hello\" },\n  { delay: 1000 }\n);\n\n// Schedule a recurring job\nqueue.schedule(\"daily-report\", { cron: \"0 0 * * *\" });\n```\n\n### Defining Workers\n\n```typescript\nimport { defineWorker } from \"plainjob\";\n\nconst worker = defineWorker(\n  \"send-email\",\n  async (job) =\u003e {\n    const { to, subject } = JSON.parse(job.data);\n    await sendEmail(to, subject);\n  },\n  {\n    queue,\n    onCompleted: (job) =\u003e console.log(`Job ${job.id} completed`),\n    onFailed: (job, error) =\u003e console.error(`Job ${job.id} failed: ${error}`),\n  }\n);\n\nworker.start();\n```\n\n### Managing Jobs\n\n```typescript\n// Count pending jobs\nconst pendingCount = queue.countJobs({ status: JobStatus.Pending });\n\n// Get job types\nconst types = queue.getJobTypes();\n\n// Get scheduled jobs\nconst scheduledJobs = queue.getScheduledJobs();\n```\n\n## Advanced Usage\n\n### Graceful Shutdown\n\nTo ensure all jobs are processed before shutting down:\n\n```typescript\nimport { processAll } from \"plainjob\";\n\nprocess.on(\"SIGTERM\", async () =\u003e {\n  console.log(\"Shutting down...\");\n  await worker.stop(); // \u003c-- finishes processing jobs\n  queue.close();\n  process.exit(0);\n});\n```\n\n### Multi-process Workers\n\nFor high-throughput scenarios, you can spawn multiple worker processes. Here's an example based on `bench-worker.ts`:\n\n```typescript\nimport { fork } from \"node:child_process\";\nimport os from \"node:os\";\n\nconst numCPUs = os.cpus().length;\nconst dbUrl = \"queue.db\";\n\nfor (let i = 0; i \u003c numCPUs; i++) {\n  const worker = fork(\"./worker.ts\", [dbUrl]);\n  worker.on(\"exit\", (code) =\u003e {\n    console.log(`Worker ${i} exited with code ${code}`);\n  });\n}\n```\n\nIn `worker.ts`:\n\n```typescript\nimport Database from \"better-sqlite3\";\nimport { better, defineQueue, defineWorker, processAll } from \"plainjob\";\n\nconst dbUrl = process.argv[2];\nconst connection = better(new Database(dbUrl));\nconst queue = defineQueue({ connection });\n\nconst worker = defineWorker(\n  \"bench\",\n  async (job) =\u003e {\n    // Process job\n  },\n  { queue }\n);\n\nvoid worker.start().catch((error) =\u003e {\n  console.error(error);\n  process.exit(1);\n});\n```\n\nThis setup allows you to leverage multiple CPU cores for processing jobs in parallel.\n\nFor more detailed information on the API and advanced usage, please refer to the source code and tests.\n","funding_links":[],"categories":["Tools"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjustplainstuff%2Fplainjob","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjustplainstuff%2Fplainjob","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjustplainstuff%2Fplainjob/lists"}