{"id":13879190,"url":"https://github.com/hlascelles/que-scheduler","last_synced_at":"2025-07-16T15:31:42.026Z","repository":{"id":26445118,"uuid":"108745688","full_name":"hlascelles/que-scheduler","owner":"hlascelles","description":"A lightweight cron scheduler for the async job worker Que","archived":false,"fork":false,"pushed_at":"2025-07-01T16:26:45.000Z","size":1939,"stargazers_count":116,"open_issues_count":8,"forks_count":23,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-07-10T09:54:50.629Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hlascelles.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-10-29T15:21:05.000Z","updated_at":"2025-07-01T16:26:40.000Z","dependencies_parsed_at":"2023-09-25T23:22:55.528Z","dependency_job_id":"0e732973-ba56-4063-875f-e5d298efcf02","html_url":"https://github.com/hlascelles/que-scheduler","commit_stats":{"total_commits":398,"total_committers":16,"mean_commits":24.875,"dds":0.6733668341708543,"last_synced_commit":"5298b079994b8dd4f7b7479ef88d79ec3191a39d"},"previous_names":[],"tags_count":47,"template":false,"template_full_name":null,"purl":"pkg:github/hlascelles/que-scheduler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hlascelles%2Fque-scheduler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hlascelles%2Fque-scheduler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hlascelles%2Fque-scheduler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hlascelles%2Fque-scheduler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hlascelles","download_url":"https://codeload.github.com/hlascelles/que-scheduler/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hlascelles%2Fque-scheduler/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264890087,"owners_count":23678833,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-06T08:02:12.813Z","updated_at":"2025-07-16T15:31:42.017Z","avatar_url":"https://github.com/hlascelles.png","language":"Ruby","funding_links":[],"categories":["Scheduling","Ruby"],"sub_categories":[],"readme":"que-scheduler\n================\n\n[![Gem Version](https://img.shields.io/gem/v/que-scheduler?color=green)](https://rubygems.org/gems/que-scheduler)\n[![specs workflow](https://github.com/hlascelles/que-scheduler/actions/workflows/specs.yml/badge.svg)](https://github.com/hlascelles/que-scheduler/actions)\n[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n[![Coverage Status](https://coveralls.io/repos/github/hlascelles/que-scheduler/badge.svg?branch=master)](https://coveralls.io/github/hlascelles/que-scheduler?branch=master)\n[![Code Climate Maintainability](https://api.codeclimate.com/v1/badges/710d2fc5202f95d76e8a/maintainability)](https://codeclimate.com/github/hlascelles/que-scheduler/maintainability)\n\n## Description\n\nque-scheduler is an extension to [Que](https://github.com/chanks/que) that adds support for scheduling \nitems using a cron style configuration file. It works by running as a que job itself, determining what \nneeds to be run, enqueueing those jobs, then enqueueing itself to check again later.\n\n## Installation\n\n1. To install, add the gem to your Gemfile:\n    ```ruby\n    gem 'que-scheduler'\n    ```\n1. Specify a schedule in a yml file or programmatically (see below). The default location that \nque-scheduler will look for it is `config/que_schedule.yml`. The format is essentially the same as\nresque-scheduler files, but with additional features.\n\n1. Add a migration to prepare the audit table and start the job scheduler. Note that this migration \n   will fail if Que is set to execute jobs synchronously, i.e. `Que::Job.run_synchronously = true`.\n\n    ```ruby\n    class CreateQueSchedulerSchema \u003c ActiveRecord::Migration[6.0]\n      def change\n        Que::Scheduler::Migrations.migrate!(version: 8)\n        Que::Scheduler::Migrations.reenqueue_scheduler_if_missing\n      end\n    end\n    ```\n    \n## Schedule configuration\n\nThe schedule file should be placed here: `config/que_schedule.yml`. Alternatively if you\nwish to generate the configuration dynamically, you can set it directly using an initializer\n(see \"Gem configuration\" below).\n\nThe file is a list of que job classes with arguments and a schedule frequency (in crontab \nsyntax). The format is similar to the resque-scheduler format, though priorities must be supplied as\nintegers, and job classes must be migrated from Resque to Que.\nActiveJob classes configured to use `:que` adapter are also supported. Cron syntax can be anything\nunderstood by [fugit](https://github.com/floraison/fugit#fugitcron).\n\nIt has one additional feature, `schedule_type: every_event`. This is set on a job that must be run for every \nsingle matching cron time that goes by, even if the system is offline over more than one match. \nTo better process these `every_event` jobs, they are always enqueued with the first \nargument being the time that they were supposed to be processed.  \n \nFor example:\n\n```yaml\nCancelAbandonedOrders:\n  cron: \"*/5 * * * *\"\n\n# Specify the job_class, using any name for the key\nqueue_documents_for_indexing:\n  cron: \"0 0 * * *\"\n  class: QueueDocuments\n  \n# Specify the job queue\nReportOrders:\n  cron: \"0 0 * * *\"\n  queue: reporting\n\n# Specify the job priority using Que's number system\nBatchOrders:\n  cron: \"0 0 * * *\"\n  priority: 25\n  \n# Specify array job arguments\nSendOrders:\n  cron: \"0 0 * * *\"\n  args: ['open']\n\n# Specify hash job arguments. Note, this appears as a single hash to `run`, not as kwargs.\nSendPreorders:\n  cron: \"0 0 * * *\"\n  args:\n    order_type: special\n  \n# Specify a single nil argument\nSendPostorders:\n  cron: \"0 0 * * *\"\n  args: ~ # See https://stackoverflow.com/a/51990876/1267203\n  \n# Use simpler cron syntax\nSendBilling:\n  cron: \"@daily\"\n  \n# Use timezone cron syntax\nSendCoupons:\n  cron: \"0 7 * * * America/Los_Angeles\"\n\n# Altogether now\nall_options_job:\n  cron: \"0 0 * * *\"\n  class: QueueDocuments\n  queue: reporting\n  priority: 25\n  args: ['open']\n  \n# Ensure you never miss a job, even after downtime, by using \"schedule_type: every_event\"\nDailyBatchReport:\n  cron: \"0 3 * * *\"\n  # This job will be run every day at 03:00 as normal.\n  # However, the \"schedule_type: every_event\" setting below will ensure that if workers are offline\n  # for any amount of time then the backlog will always be enqueued on recovery.\n  # See \"Schedule types\" below for more information.\n  schedule_type: every_event\n```\n\n## Schedule types\n\nA job can have a `schedule_type` assigned to it. Valid values are:\n\n1. `default` - This job will be scheduled in a manner closer to resque-scheduler. If multiple cron\n  times go by during an extended period of downtime (eg a long maintenance window) then only one job\n  will be enqueued when the system starts back up. Multiple missed events are coalesced. This mimics\n  the way resque-scheduler would perform if it were taken down for some time.\n1. `every_event` - Every cron match will result in a job being scheduled. If multiple cron times go \n  by during an extended period of downtime, then a job will be scheduled for every one missed on \n  startup. This `schedule_type` should be used for regular batch jobs that need to know which time\n  they are running a batch for. The job will always be scheduled with an ISO8601 string of the cron \n  that matched as the first argument. \n  \n  An example would be an eventing DailyReportJob which summarises a day's sales. If no jobs run for\n  a few days due to a technical failure, then on recovery a report would still be needed for each \n  individual day. \"schedule_type: every_event\" would ensure this happens.\n  \n  This feature ensures that jobs which *must run* for a certain cron match will always eventually \n  execute, even after a total system crash, or even a DB backup restore.\n\n## Gem configuration\n\nYou can configure some aspects of the gem with a config block (eg in a Rails initializer). \nThe default is given below. You can omit any configuration sections you are not intending to change.\nIt is quite likely you won't have to create this config at all.\n\n```ruby\nQue::Scheduler.configure do |config|\n  # The location of the schedule yaml file.\n  config.schedule_location = ENV.fetch(\"QUE_SCHEDULER_CONFIG_LOCATION\", \"config/que_schedule.yml\")\n\n  # The schedule as a hash. You can use this if you want to build the schedule yourself at runtime.\n  # This will override the above value if provided.\n  config.schedule = {\n    SpecifiedByHashTestJob: {\n      cron: \"02 11 * * *\"\n    }\n  }\n  \n  # The transaction block adapter. By default, que-scheduler uses the one supplied by que.\n  # However if, for example, you rely on listeners to ActiveRecord's exact `transaction` method, or \n  # Sequel's DB.after_commit helper, then you can supply it here.\n  config.transaction_adapter = ::Que.method(:transaction)\n\n  # Which queue name the que-scheduler job should self-schedule on. Typically this is the default\n  # queue of que, which has a different name in Que 0.x (\"\") and 1.x (\"default\").\n  # It *must* be the \"highest throughput\" queue - do not work the scheduler on a \"long \n  # running jobs\" queue. It is very unlikely you will want to change this. \n  config.que_scheduler_queue = ENV.fetch(\"QUE_SCHEDULER_QUEUE\", \"\" or \"default\")\n  \n  # If que-scheduler is being used with Rails, then it will inherit the time zone from that \n  # framework, and you can leave the value as nil as shown below. However, if you are not using\n  # Rails, you may need to set the time zone here. If que-scheduler cannot determine the time zone\n  # it will yield an error prompting you for action.\n  # If you need to set a value, use the string representation:\n  # eg: config.time_zone = \"Europe/London\"\n  config.time_zone = nil\nend\n```\n\n## Scheduler Audit\n\nAn audit table `que_scheduler_audit` is written to by the scheduler to keep a history of when the \nscheduler ran to calculate what was necessary to run (if anything). It is created by the included \nmigration tasks.\n\nAdditionally, there is the audit table `que_scheduler_audit_enqueued`. This logs every job that \nthe scheduler enqueues.\n\nque-scheduler comes with the `QueSchedulerAuditClearDownJob` job built in that you can optionally\nschedule to clear down audit rows if you don't need to retain them indefinitely. You should add this\nto your own scheduler config yaml.\n\nFor example:\n\n```yaml\n# This will clear down the oldest que-scheduler audit rows. Since que-scheduler\n# runs approximately every minute, 129600 is 90 days.\nQue::Scheduler::Jobs::QueSchedulerAuditClearDownJob:\n  cron: \"0 0 * * *\"\n  args:\n    retain_row_count: 129600\n```\n\n## Required migrations\n\nWhen there is a major version (breaking) change, a migration should be run in. The version of the \nlatest migration proceeds at a faster rate than the version of the gem. eg If the gem is on version\n3 then the migrations may be on version 6). \n\nTo run in all the migrations required up to a number, just migrate to that number with one line, and\nit will perform all the intermediary steps. \n\nie, This will perform all migrations necessary up to the latest version, skipping any already \nperformed.\n\n```ruby\nclass CreateQueSchedulerSchema \u003c ActiveRecord::Migration[6.0]\n  def change\n    Que::Scheduler::Migrations.migrate!(version: 8)\n    Que::Scheduler::Migrations.reenqueue_scheduler_if_missing\n  end\nend\n```\n\nThe changes in past migrations were: \n\n| Version | Changes                                                                             |\n|:-------:|-------------------------------------------------------------------------------------|\n|    1    | Enqueued the main Que::Scheduler. This is the job that performs the scheduling.     |\n|    2    | Added the audit table `que_scheduler_audit`.                                        |\n|    3    | Added the audit table `que_scheduler_audit_enqueued`.                               |\n|    4    | Updated the the audit tables to use bigints                                         |\n|    5    | Dropped an unnecessary index                                                        |\n|    6    | Enforced single scheduler job at the trigger level                                  |\n|    7    | Prevent accidental deletion of scheduler job                                        |\n|    8    | Add primary key to audit. Note, this can be a slow migration if you have many rows! |\n\nThe changes to the DB ([DDL](https://en.wikipedia.org/wiki/Data_definition_language)) are all \ncaptured in the structure.sql so will be re-run in correctly if squashed - except for the actual \nscheduling of the job itself (as that is [DML](https://en.wikipedia.org/wiki/Data_manipulation_language)).\nIf you squash your migrations make sure this is added as the final line:\n\n```ruby\nQue::Scheduler::Migrations.reenqueue_scheduler_if_missing\n```\n\n## HA Redundancy and DB restores\n\nBecause of the way que-scheduler works, it requires no additional processes. It is, itself, a Que job.\nAs long as there are Que workers functioning, then jobs will continue to be scheduled correctly. There\nare no HA concerns to worry about and no namespace collisions between different databases. \n\nAdditionally, like Que, when your database is backed up, your scheduling state is stored too. If your \nworkers are down for an extended period, or a DB restore is performed, the scheduler will always be \nin a coherent state with the rest of your database.\n\n## Concurrent scheduler detection\n\nNo matter how many tasks you have defined in your schedule, you will only ever need one que-scheduler\njob enqueued. que-scheduler knows this, and there are DB constraints in place to ensure there is\nonly ever exactly one scheduler job.\n\nIt also follows que job design [best practices](https://github.com/chanks/que/blob/master/docs/writing_reliable_jobs.md),\nusing ACID guarantees, to ensure that it will never run multiple times. If the scheduler crashes for any reason,\nit will rollback correctly and try again. It won't schedule jobs twice for a cron match.\n\n## How it works\n\nque-scheduler is a job that reads a schedule file, enqueues any jobs it determines that need to be run,\nthen reschedules itself. The flow is as follows:\n\n1. The que-scheduler job runs for the very first time.\n1. que-scheduler loads the schedule file. It will not schedule any other jobs, except itself, \n   as it has never run before.\n1. Some time later it runs again. It knows what jobs it should be monitoring, and notices that some \n   have are due. It enqueues those jobs and then itself. Repeat.\n1. After a deploy that changes the schedule, the job notices any new jobs to schedule, and knows which\n   ones to forget. It does not need to be re-enqueued or restarted.\n   \n## Testing Configuration\n\nYou can add tests to validate your configuration during the spec phase. This will perform a variety \nof sanity checks and ensure that:\n\n1. The yml is present and valid\n1. The job classes exist and are descendants of Que::Job\n1. The cron fields are present and valid\n1. The queues (if present) are strings\n1. The priorities (if present) are integers\n1. The schedule_types are known\n\n```ruby\n\n  describe 'check que_schedule.yml' do\n    it 'loads the schedule from the default location' do\n      # Will raise an error if any config is invalid\n      expect(Que::Scheduler.schedule).not_to be nil\n    end\n  end\n```\n\n## Error Notification\n\nIf there is an error during scheduling, que-scheduler will report it using the [standard que error\nnotifier](https://github.com/chanks/que/blob/master/docs/error_handling.md#error-notifications).\nThe scheduler will then continue to retry indefinitely.\n\n## Upgrading\n\nque-scheduler uses [semantic versioning](https://semver.org/), so major version changes will usually \nrequire additional actions to be taken upgrading from one major version to another. \n\n## Changelog\n\nA full changelog can be found here: [CHANGELOG.md](https://github.com/hlascelles/que-scheduler/blob/master/CHANGELOG.md)\n\n## System requirements\n\nYour [postgres](https://www.postgresql.org/) database must be at least version 9.6.0.\n\nThe latest version of que-scheduler supports Ruby 3.0 and above.\nque-scheduler versions below 4.4.0 work with Ruby 2.7.\nque-scheduler versions below 4.2.3 work with Ruby 2.5 and Ruby 2.6.\n\n## Inspiration\n\nThis gem was inspired by the makers of the excellent [Que](https://github.com/chanks/que) job scheduler gem. \n\n## Contributors\n\n* @bnauta\n* @bjeanes\n* @JackDanger\n* @jish\n* @joehorsnell\n* @krzyzak\n* @papodaca\n* @ajoneil\n* @ippachi\n* @milgner\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhlascelles%2Fque-scheduler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhlascelles%2Fque-scheduler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhlascelles%2Fque-scheduler/lists"}