{"id":7277482,"url":"https://github.com/joseferben/pocketbase-queue","last_synced_at":"2025-07-08T00:05:39.344Z","repository":{"id":229801057,"uuid":"777663617","full_name":"joseferben/pocketbase-queue","owner":"joseferben","description":"A type-safe queue for background tasks using PocketBase.","archived":false,"fork":false,"pushed_at":"2024-03-31T17:16:05.000Z","size":198,"stargazers_count":37,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-12-11T06:37:57.304Z","etag":null,"topics":["background-jobs","pocketbase","queue","typescript","workers"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joseferben.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-03-26T09:31:56.000Z","updated_at":"2024-11-30T21:03:36.000Z","dependencies_parsed_at":"2024-03-31T18:24:57.506Z","dependency_job_id":null,"html_url":"https://github.com/joseferben/pocketbase-queue","commit_stats":null,"previous_names":["joseferben/pocketbase-queue"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joseferben%2Fpocketbase-queue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joseferben%2Fpocketbase-queue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joseferben%2Fpocketbase-queue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joseferben%2Fpocketbase-queue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joseferben","download_url":"https://codeload.github.com/joseferben/pocketbase-queue/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230105974,"owners_count":18173971,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["background-jobs","pocketbase","queue","typescript","workers"],"created_at":"2024-04-03T06:24:47.609Z","updated_at":"2024-12-17T11:18:24.736Z","avatar_url":"https://github.com/joseferben.png","language":"TypeScript","funding_links":[],"categories":["Tools/Plugin","Other tools"],"sub_categories":["Other"],"readme":"# pocketbase-queue\n\nA type-safe queue for background tasks on top of PocketBase. Works in all JavaScript environments where the PocketBase JS SDK is supported.\n\nThis is **not** a high-throughput queue, but it's a good solution to keep things simple. It works with a vanilla PocketBase installation, no changes or additional hooks needed.\n\n```typescript\nimport { createConnection, createQueue } from \"pocketbase-queue\";\n\nconst pb = ... // PocketBase instance\nconst connection = await createConnection({ pb });\n\nconst queue = createQueue\u003c{ message: string }\u003e({\n  name: \"greeting\",\n  connection,\n});\n\nqueue.push({ message: \"Hello, world!\" });\n\nqueue.process({ concurrency: 2 }, async ({ task }) =\u003e {\n  console.log(task.message);\n});\n```\n\n![Screenshot](/screenshot.png)\n\n## Installation\n\n```bash\nnpm i pocketbase pocketbase-queue\n```\n\nIn order to use CJS instead of ESM, import the package like this:\n\n```typescript\nimport { createConnection, createQueue } from \"pocketbase-queue/cjs\";\n```\n\nImport the queue collections to your PocketBase instance. Don't forget to tick `Merge with the existing collections`:\n\n```json\n[\n  {\n    \"id\": \"v2i9s97ijtkb6as\",\n    \"name\": \"queue_locks\",\n    \"type\": \"base\",\n    \"system\": false,\n    \"schema\": [\n      {\n        \"system\": false,\n        \"id\": \"gb9nqipq\",\n        \"name\": \"worker_id\",\n        \"type\": \"text\",\n        \"required\": true,\n        \"presentable\": false,\n        \"unique\": false,\n        \"options\": {\n          \"min\": null,\n          \"max\": null,\n          \"pattern\": \"\"\n        }\n      },\n      {\n        \"system\": false,\n        \"id\": \"of2tn7ct\",\n        \"name\": \"task\",\n        \"type\": \"relation\",\n        \"required\": true,\n        \"presentable\": false,\n        \"unique\": false,\n        \"options\": {\n          \"collectionId\": \"qgf0f8rtdk0s6pq\",\n          \"cascadeDelete\": false,\n          \"minSelect\": null,\n          \"maxSelect\": 1,\n          \"displayFields\": null\n        }\n      }\n    ],\n    \"indexes\": [\n      \"CREATE UNIQUE INDEX `idx_SKtU4DZ` ON `queue_locks` (`task`)\",\n      \"CREATE INDEX `idx_GpAPmOu` ON `queue_locks` (`created`)\"\n    ],\n    \"listRule\": null,\n    \"viewRule\": null,\n    \"createRule\": null,\n    \"updateRule\": null,\n    \"deleteRule\": null,\n    \"options\": {}\n  },\n  {\n    \"id\": \"qgf0f8rtdk0s6pq\",\n    \"name\": \"queue_tasks\",\n    \"type\": \"base\",\n    \"system\": false,\n    \"schema\": [\n      {\n        \"system\": false,\n        \"id\": \"kdzlqnxa\",\n        \"name\": \"queue\",\n        \"type\": \"text\",\n        \"required\": true,\n        \"presentable\": false,\n        \"unique\": false,\n        \"options\": {\n          \"min\": null,\n          \"max\": null,\n          \"pattern\": \"\"\n        }\n      },\n      {\n        \"system\": false,\n        \"id\": \"odvdwps6\",\n        \"name\": \"task\",\n        \"type\": \"json\",\n        \"required\": false,\n        \"presentable\": false,\n        \"unique\": false,\n        \"options\": {\n          \"maxSize\": 2000000\n        }\n      },\n      {\n        \"system\": false,\n        \"id\": \"lfsvih6z\",\n        \"name\": \"failed\",\n        \"type\": \"date\",\n        \"required\": false,\n        \"presentable\": false,\n        \"unique\": false,\n        \"options\": {\n          \"min\": \"\",\n          \"max\": \"\"\n        }\n      },\n      {\n        \"system\": false,\n        \"id\": \"h530uquw\",\n        \"name\": \"failed_reason\",\n        \"type\": \"text\",\n        \"required\": false,\n        \"presentable\": false,\n        \"unique\": false,\n        \"options\": {\n          \"min\": null,\n          \"max\": null,\n          \"pattern\": \"\"\n        }\n      }\n    ],\n    \"indexes\": [\n      \"CREATE INDEX `idx_jyHoldi` ON `queue_tasks` (\\n  `queue`,\\n  `failed`,\\n  `created`\\n)\",\n      \"CREATE INDEX `idx_TJTrLZe` ON `queue_tasks` (\\n  `failed`,\\n  `updated`\\n)\"\n    ],\n    \"listRule\": null,\n    \"viewRule\": null,\n    \"createRule\": null,\n    \"updateRule\": null,\n    \"deleteRule\": null,\n    \"options\": {}\n  },\n  {\n    \"id\": \"ronp4q37zviiby1\",\n    \"name\": \"queue_stats\",\n    \"type\": \"view\",\n    \"system\": false,\n    \"schema\": [\n      {\n        \"system\": false,\n        \"id\": \"an0kocfc\",\n        \"name\": \"pending_tasks\",\n        \"type\": \"json\",\n        \"required\": false,\n        \"presentable\": false,\n        \"unique\": false,\n        \"options\": {\n          \"maxSize\": 1\n        }\n      },\n      {\n        \"system\": false,\n        \"id\": \"nnwglxhx\",\n        \"name\": \"failed_tasks\",\n        \"type\": \"json\",\n        \"required\": false,\n        \"presentable\": false,\n        \"unique\": false,\n        \"options\": {\n          \"maxSize\": 1\n        }\n      }\n    ],\n    \"indexes\": [],\n    \"listRule\": null,\n    \"viewRule\": null,\n    \"createRule\": null,\n    \"updateRule\": null,\n    \"deleteRule\": null,\n    \"options\": {\n      \"query\": \"SELECT \\n  queue as id,\\n  SUM(CASE WHEN failed IS '' THEN 1 ELSE 0 END) as pending_tasks,\\n  SUM(CASE WHEN failed IS NOT '' THEN 1 ELSE 0 END) as failed_tasks\\nFROM queue_tasks\\nGROUP BY queue;\"\n    }\n  }\n]\n```\n\n## Usage\n\n```typescript\nimport { createConnection, createQueue } from \"pocketbase-queue\";\n\n// Create a service admin user and use its credentials to create a connection:\nconst pb = new Pocketbase(process.env.POCKETBASE_URL || \"http://127.0.0.1:8090\");\nawait pb.admins.authWithPassword(\n  process.env.POCKETBASE_EMAIL, // Email of a PocketBase Admin\n  process.env.POCKETBASE_PASSWORD // Password of the PocketBase Admin\n);\n\nconst connection = await createConnection({ pb, verbose: true });\n\nconst queue = createQueue\u003c{ message: string }\u003e({\n  name: \"greeting\",\n  connection,\n});\n\nqueue.push({ message: \"Hello, world!\" });\n\nqueue.process({ concurrency: 2 }, async ({ task }) =\u003e {\n  console.log(task.message);\n});\n```\n\n## Advanced usage\n\n```typescript\nqueue.on(\"error\", (error) =\u003e {\n  // Prints pocketbase-queue errors, actual task errors are handled by the task processor\n  console.error(error);\n});\n\nqueue.on(\"stats\", (stats) =\u003e {\n  // Prints basic queue stats, like tasks per second\n  console.log(stats);\n});\n\n// On Node, use this to gracefully exit workers. This makes sure there are no unreleased locks.\n// Unreleased locks get cleaned up after 5 minutes, but it's better to release them as soon as possible.\nprocess.on(\"SIGINT\", async () =\u003e {\n  queue.close();\n  await new Promise((resolve) =\u003e setImmediate(resolve));\n  console.log(\"exiting\");\n  process.exit(0);\n});\n```\n\n## Run tests\n\nInstall and start PocketBase:\n\n```bash\n./pocketbase/pocketbase serve --dev\n```\n\nRun the tests:\n\n```bash\nPOCKETBASE_EMAIL=\u003cadmin email\u003e POCKETBASE_PASSWORD=\u003cadmin password\u003e npm run dev\n```\n\n## Considerations\n\n- I easily reached 50-60 tasks per second with 4 concurrent workers and a local PocketBase instance\n- Make sure to run the workers as close as possible to the PocketBase instance to reduce latency\n- Failed tasks are stored with the error message for 7 days, use `failedTaskTtl` to configure this setting in milliseconds\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoseferben%2Fpocketbase-queue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoseferben%2Fpocketbase-queue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoseferben%2Fpocketbase-queue/lists"}