{"id":21665853,"url":"https://github.com/totaltechgeek/pg-swag","last_synced_at":"2026-02-23T11:14:20.916Z","repository":{"id":247258631,"uuid":"825384726","full_name":"TotalTechGeek/pg-swag","owner":"TotalTechGeek","description":"Distributed Scheduling with a Grin ","archived":false,"fork":false,"pushed_at":"2025-01-16T17:31:27.000Z","size":304,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-17T01:50:59.435Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TotalTechGeek.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-07-07T16:13:49.000Z","updated_at":"2025-01-16T17:31:28.000Z","dependencies_parsed_at":"2025-01-14T19:39:17.636Z","dependency_job_id":"3dfd634f-6c12-484e-b78e-9d8445832f14","html_url":"https://github.com/TotalTechGeek/pg-swag","commit_stats":null,"previous_names":["totaltechgeek/pg-swag"],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/TotalTechGeek/pg-swag","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TotalTechGeek%2Fpg-swag","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TotalTechGeek%2Fpg-swag/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TotalTechGeek%2Fpg-swag/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TotalTechGeek%2Fpg-swag/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TotalTechGeek","download_url":"https://codeload.github.com/TotalTechGeek/pg-swag/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TotalTechGeek%2Fpg-swag/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264023553,"owners_count":23545689,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-25T11:18:08.854Z","updated_at":"2026-02-23T11:14:20.869Z","avatar_url":"https://github.com/TotalTechGeek.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scheduling with a Grin\n\n[![Coverage Status](https://coveralls.io/repos/github/TotalTechGeek/pg-swag/badge.svg?branch=main)](https://coveralls.io/github/TotalTechGeek/pg-swag?branch=main)\n\n\u003cimg src=\"https://i.imgur.com/Xio7kHx.png\" alt=\"An elephant wearing sunglasses\" width=\"250px\" /\u003e\n\nHave you ever needed to run a task (emails, reports, cleanup) at a specific time, or on a recurring schedule? Have you ever had multiple processes you needed to distribute the workload between?\n\nPG Swag is a distributed scheduling library intended to simplify the process of scheduling tasks across one or more nodes, leveraging the Postgres database as a shared state.\n\nDuring our local testing, we found that this setup easily scaled to 30K Recurring Schedules/s on a single node; however, we believe it to be unlikely that the scheduler will be your bottleneck.\n\n### Information\n\nIn this package, there are no \"leader nodes\", each node is responsible for grabbing and executing tasks. It also does not require a separate service to manage the scheduling, as the database is used as the source of truth. Additionally, it is designed to be resilient to node failures, and can be run in a distributed environment. Lastly, it is designed to be efficient with fetching tasks -- it precomputes when the task should be run, and only fetches tasks that are ready to be run.\n\nThe querying leverages skip locks to ensure that only one node is running a task at a time; however, I believe it would be trivial to remove it and apply this system to other databases.\n\n### How to use\n\nTo use this package, you will need to install it with your preferred package manager; we enjoy bun:\n\n```bash\nbun add pg-swag\n```\n\nThen, you can use it in your code like so:\n\n```javascript\nimport { Swag } from 'pg-swag';\n\nconst swag = new Swag({\n    dialect: 'postgres',\n    config: {\n        host: 'localhost',\n        user: 'postgres',\n        password: 'password',\n    }\n})\n\nawait swag.on('email', async job =\u003e {\n    console.log('Sending email to', job.data.email, 'with message', job.data.body)\n})\n\n// Schedules an email to be sent to Bob every day from July 1st, 2024\nawait swag.schedule('email', { \n    email: 'bob@example.com',\n    body: 'Hello, Bob!'\n}, 'R/2024-07-01/P1D')\n```\n\nAlternatively, you can pass in a query method like so:\n\n```javascript\nimport { Swag } from 'pg-swag';\n\nconst swag = new Swag({\n    dialect: 'postgres',\n    query: db.query \n})\n```\n\n### Supported Scheduling\n\nWe support a variety of scheduling options, including:\n\n- [ISO 8601 Repeating Intervals](https://en.wikipedia.org/wiki/ISO_8601#Repeating_intervals)\n- [ISO 8601 Dates and Times](https://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations)\n- [Cron Expressions](https://en.wikipedia.org/wiki/Cron)\n\nSome examples are outlined below.\n\n#### ISO 8601 Repeating Intervals\n\nThese are defined by the `R` prefix, followed by the start date, and then the interval. For example, `R/2024-07-01/P1D` would start on July 1st, 2024, and repeat every day.\n\n`P` represents the period, and can be followed by `Y` for years, `M` for months, `W` for weeks, and `D` for days. Using `T` will allow you to specify hours, minutes, and seconds.\n\nExamples:\n\n- `R/2024-07-01/P1D` - Every day starting July 1st, 2024\n- `R/2024-07-01T12:00:00/PT1H` - Every hour starting July 1st, 2024 at 12:00:00\n- `R/2024-07-01T12:00:00/PT1H30M` - Every hour and a half starting July 1st, 2024 at 12:00:00\n- `R/PT1H` - Every hour starting now\n\nIf you specify a start date, you can also specify the number of recurrences you'd like (`R#`):\n\n- `R5/2024-07-01/P1D` - Every day starting July 1st, 2024, up to 5 times\n- `R5/2024-07-01T12:00:00/PT1H` - Every hour starting July 1st, 2024 at 12:00:00, up to 5 times\n\nAdditionally, we support one non-standard format for this interval, `R/\u003cstart\u003e/\u003cend\u003e/\u003cinterval\u003e`:\n\n- `R/2024-07-01/2024-07-05/P1D` - Every day from July 1st, 2024 to July 5th, 2024\n- `R/2024-07-01T12:00:00/2024-07-01T13:00:00/PT1M` - Every minute from July 1st, 2024 at 12:00:00 to July 1st, 2024 at 13:00:00\n\n#### ISO 8601 Dates and Times\n\n- `2024-07-01T12:00:00` - July 1st, 2024 at 12:00:00\n- `2020-01-01` - January 1st, 2020\n- `new Date('2020-01-01')` - JavaScript Dates are also supported\n\nThese will not repeat, and will only run once.\n\n#### Duration Objects\n\nYou can also pass in a duration object, which is an object with the following fields:\n\nField | Description\n-- | --\nyears | # of Years\nmonths | # of Months\nweeks | # of Weeks\ndays | # of Days\nhours | # of Hours\nminutes | # of Minutes\nseconds | # of Seconds\nrecurrences | # of Recurrences\nstartDate | The date to start the schedule\nendDate | The date to end the schedule\n\nUpon calling `schedule`, the duration object will be converted into an ISO8601 Repeating Interval.\n\n- `{ days: 1 }` - Every day starting now\n- `{ days: 1, recurrences: 5 }` - Every day starting now, up to 5 times\n- `{ hours: 1, startDate: '2024-07-01T12:00:00' }` - Every hour starting July 1st, 2024 at 12:00:00\n\n#### Cron Expressions\n\nWe use the [cron-parser](https://www.npmjs.com/package/cron-parser) package to parse cron expressions. These are defined by the standard cron syntax (and we specifically recommend the 5 field syntax). For example, `0 0 * * *` would run every day at midnight.\n\n#### Future Additions\n\nWe are planning on adding support for more scheduling options, for example: \"every sunrise\" or \"every 3 sunrises\". If someone needs this, file an issue and we will prioritize it.\n\n### Technical Details\n\nIn this package, jobs \u0026 schedules are one and the same; there is no distinction between the two, thus allowing us to leverage a single table design.\n\nThe table structure is as follows\n\nField | Description\n-- | --\nqueue | The type of task you wish to perform\nid | An ID to represent the task\nrun_at | Generated field to determine when the task should be run\ndata | The data to be passed to the task\nexpression | The scheduling expression\nlocked_until | The time the task is locked until\nlocked_by | The node that has locked the task\nattempts | The number of attempts that have been made to run the task\n\nThe main index for fetching tasks is on `queue` and `greatest(run_at, locked_until)` to make it efficient to fetch tasks that are ready to be run.\n\nWe also have a unique index on `queue` and `id` to ensure that tasks are not duplicated, and make it efficient to update \u0026 delete tasks.\n\n**Note:** In the future, it might be wise for us to automatically handle partitioning the table as different queues are introduced and such. It does not at the moment, however, we do not anticipate this being a problem for most users. It should be reasonably performant for hundreds of thousands of tasks (your bottleneck will likely not be the scheduler).\n\n### Class Configuration\n\nWhen creating a new instance of the scheduler, there are two main options to pass in:\n\nOption | Description | Default\n-- | -- | --\nConnection Configuration | [The configuration for the Postgres connection](https://github.com/vitaly-t/pg-promise/wiki/Connection-Syntax) | {}\nTable Configuration | { schema?: null \\| string, table?: string } | { table: 'jobs', schema: null }\n\nBy default, it will create a table called `jobs` in the public schema. If you want to use a different schema or table name, you can pass it in as an option.\n\n### Schedule Configuration\n\nWhen setting up a reader for a queue, you can pass in a configuration object to customize the behavior of the reader. The following options are available:\n\nOption | Description | Type | Default\n-- | -- | -- | --\nbatchSize | The number of tasks to fetch at a time | number | 100\nconcurrentJobs | The number of tasks to run concurrently | number | 10\npollingPeriod | The amount of time to wait between polling for tasks | number (milliseconds) or string ('15 seconds') | 15000\nflushPeriod | The amount of time before writing finished tasks to the database |  number (milliseconds) or string ('15 seconds')  | 1000\nlockPeriod | The amount of time to lock a task for | number (milliseconds) or string ('15 seconds')  | '1 minutes'\nskipPast | Whether to schedule tasks in the past, or to continue the schedule after current time. See below. | boolean | true\nmaxHeartbeats | The number of heartbeats before releasing a lock on tasks not actively being worked on in a batch | number | Infinity\n\n#### Skip Past\n\nBy default, the scheduler will not schedule tasks in the past. For example, if you have a 5 minute periodicity, and your service was down for 6h, when the scheduler starts back up, it will schedule 5m from the current time, not 6h ago, this is to avoid running ~72 tasks.\n\nIf you want to run all the tasks that were missed, you can set `skipPast` to `false`. This will schedule all the tasks that were missed, and then continue the schedule as normal.\n\n### Error Handling\n\nYou might notice that the module does not have an option like `maxAttempts`. This is because we believe that the error handling should be done in the task (or its error handler) itself. If you want to retry a task, you can simply throw an error, and the task will be retried.\n\nYou can also return an object from a job to modify certain behaviors. For example, you can return `{ expression: 'cancel' }` to cancel the task.\n\nThe handler will receive the number of attempts that have been made, and you can use this to determine if you should retry the task.\n\n```javascript\nswag.on('email', async job =\u003e {\n    if (job.attempts \u003e 3) return { expression: 'cancel' }\n    // ...\n})\n```\n\nIf you wanted, you could shift this behavior into the error handler for a queue,\n\n```javascript\nswag.on('email', async job =\u003e {\n    // ...\n}).onError(async (err, job) =\u003e {\n    if (job.attempts \u003e 3) return { expression: 'cancel' }\n})\n```\n\nOr apply it globally, for all queues,\n\n```javascript\nswag.onError(async (err, job) =\u003e {\n    if (job.attempts \u003e 3) return { expression: 'cancel' }\n})\n```\n\nWe've provided a utility function to make this simpler:\n\n```javascript\nimport { cancelAfter, Swag } from 'pg-swag'\n// ... \nSwag.onError(cancelAfter(3))\n```\n\nHowever, we STRONGLY advise that you go beyond using `cancelAfter` and write some more sophisticated error handling logic, so that you can communicate to the user (or your developers) what went wrong.\n\nSimilarly, it is possible to use `{ lockedUntil: Date }` to lock the task until a specific time. This can be useful if you want to programmatically delay a task upon failure.\n\n```javascript\nSwag.onError(async (err, job) =\u003e {\n    // Lock the task by 1 extra minute for each attempt\n    return { lockedUntil: new Date(Date.now() + 1000 * 60 * job.attempts) }\n})\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftotaltechgeek%2Fpg-swag","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftotaltechgeek%2Fpg-swag","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftotaltechgeek%2Fpg-swag/lists"}