{"id":13485884,"url":"https://github.com/kagkarlsson/db-scheduler","last_synced_at":"2025-03-27T19:31:53.082Z","repository":{"id":36311540,"uuid":"40616171","full_name":"kagkarlsson/db-scheduler","owner":"kagkarlsson","description":"Persistent cluster-friendly scheduler for Java","archived":false,"fork":false,"pushed_at":"2025-03-17T12:30:00.000Z","size":2917,"stargazers_count":1341,"open_issues_count":90,"forks_count":207,"subscribers_count":19,"default_branch":"master","last_synced_at":"2025-03-22T08:01:51.691Z","etag":null,"topics":["db-scheduler","hacktoberfest","java","jdbc","job-scheduler","scheduler","spring-boot","task-scheduler"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kagkarlsson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["kagkarlsson"]}},"created_at":"2015-08-12T18:01:34.000Z","updated_at":"2025-03-19T11:44:04.000Z","dependencies_parsed_at":"2023-10-17T01:59:31.812Z","dependency_job_id":"123c522c-942a-4b70-af56-241bc45d7c22","html_url":"https://github.com/kagkarlsson/db-scheduler","commit_stats":null,"previous_names":[],"tags_count":78,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kagkarlsson%2Fdb-scheduler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kagkarlsson%2Fdb-scheduler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kagkarlsson%2Fdb-scheduler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kagkarlsson%2Fdb-scheduler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kagkarlsson","download_url":"https://codeload.github.com/kagkarlsson/db-scheduler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245910943,"owners_count":20692520,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["db-scheduler","hacktoberfest","java","jdbc","job-scheduler","scheduler","spring-boot","task-scheduler"],"created_at":"2024-07-31T18:00:33.383Z","updated_at":"2025-03-27T19:31:53.066Z","avatar_url":"https://github.com/kagkarlsson.png","language":"Java","funding_links":["https://github.com/sponsors/kagkarlsson"],"categories":["Projects","项目","任务调度","hacktoberfest","Java","\u003ca name=\"Java\"\u003e\u003c/a\u003eJava"],"sub_categories":["Job Scheduling","作业调度"],"readme":"# db-scheduler\n\n![build status](https://github.com/kagkarlsson/db-scheduler/workflows/build/badge.svg)\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.github.kagkarlsson/db-scheduler/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.github.kagkarlsson/db-scheduler)\n[![License](http://img.shields.io/:license-apache-brightgreen.svg)](http://www.apache.org/licenses/LICENSE-2.0.html)\n\nTask-scheduler for Java that was inspired by the need for a clustered `java.util.concurrent.ScheduledExecutorService` simpler than Quartz.\n\nAs such, also appreciated by users ([cbarbosa2](https://github.com/kagkarlsson/db-scheduler/issues/115#issuecomment-649601944), [rafaelhofmann](https://github.com/kagkarlsson/db-scheduler/issues/140#issuecomment-704955500), [BukhariH](https://github.com/kagkarlsson/db-scheduler/pull/268#issue-1147378003)):\n\n\u003e Your lib rocks! I'm so glad I got rid of Quartz and replaced it by yours which is way easier to handle!\n\u003e\n\u003e [cbarbosa2](https://github.com/cbarbosa2)\n\nSee also [why not Quartz?](#why-db-scheduler-when-there-is-quartz)\n\n## Features\n\n* **Cluster-friendly**. Guarantees execution by single scheduler instance.\n* **Persistent** tasks. Requires a _single_ database-table for persistence.\n* **Embeddable**. Built to be embedded in existing applications.\n* **High throughput**. Tested to handle 2k - 10k executions / second. [Link](#benchmark-test).\n* **Simple**.\n* **Minimal dependencies**. (slf4j)\n\n## Table of contents\n\n* [Getting started](#getting-started)\n* [Who uses db-scheduler?](#who-uses-db-scheduler)\n* [Examples](#examples)\n* [Configuration](#configuration)\n* [Third-party extensions](#third-party-extensions)\n* [Spring Boot usage](#spring-boot-usage)\n* [Interacting with scheduled executions using the SchedulerClient](#interacting-with-scheduled-executions-using-the-schedulerclient)\n* [How it works](#how-it-works)\n* [Performance](#performance)\n* [Versions / upgrading](#versions--upgrading)\n* [FAQ](#faq)\n\n## Getting started\n\n1. Add maven dependency\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.github.kagkarlsson\u003c/groupId\u003e\n    \u003cartifactId\u003edb-scheduler\u003c/artifactId\u003e\n    \u003cversion\u003e15.1.1\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n2. Create the `scheduled_tasks` table in your database-schema. See table definition for [postgresql](db-scheduler/src/test/resources/postgresql_tables.sql), [oracle](db-scheduler/src/test/resources/oracle_tables.sql), [mssql](db-scheduler/src/test/resources/mssql_tables.sql) or [mysql](db-scheduler/src/test/resources/mysql_tables.sql).\n\n3. Instantiate and start the scheduler, which then will start any defined recurring tasks.\n\n```java\nRecurringTask\u003cVoid\u003e hourlyTask = Tasks.recurring(\"my-hourly-task\", FixedDelay.ofHours(1))\n        .execute((inst, ctx) -\u003e {\n            System.out.println(\"Executed!\");\n        });\n\nfinal Scheduler scheduler = Scheduler\n        .create(dataSource)\n        .startTasks(hourlyTask)\n        .threads(5)\n        .build();\n\n// hourlyTask is automatically scheduled on startup if not already started (i.e. exists in the db)\nscheduler.start();\n```\n\nFor more examples, continue reading. For details on the inner workings, see [How it works](#how-it-works). If you have a Spring Boot application, have a look at [Spring Boot Usage](#spring-boot-usage).\n\n## Who uses db-scheduler?\n\nList of organizations known to be running db-scheduler in production:\n\n| Company                                   | Description                                                  |\n|-------------------------------------------|--------------------------------------------------------------|\n| [Digipost](https://digipost.no)           | Provider of digital mailboxes in Norway                      |\n| [Vy Group](https://www.vy.no/en)          | One of the largest transport groups in the Nordic countries. |\n| [Wise](https://wise.com/)                 | A cheap, fast way to send money abroad.                      |\n| Becker Professional Education             |                                                              |\n| [Monitoria](https://monitoria.ca)         | Website monitoring service.                                  |\n| [Loadster](https://loadster.app)          | Load testing for web applications.                           |\n| [Statens vegvesen](https://www.vegvesen.no/)| The Norwegian Public Roads Administration                  |\n| [Lightyear](https://lightyear.com/)       |  A simple and approachable way to invest your money globally.|\n| [NAV](https://www.nav.no/)                |  The Norwegian Labour and Welfare Administration             |\n| [ModernLoop](https://modernloop.io/)      |  Scale with your company’s hiring needs by using ModernLoop to increase efficiency in interview scheduling, communication, and coordination.             |\n| [Diffia](https://www.diffia.com/)         |  Norwegian eHealth company                                   |\n| [Swan](https://www.swan.io/)              | Swan helps developers to embed banking services easily into their product. |\n| [TOMRA](https://www.tomra.com/)           | TOMRA is a Norwegian multinational company that designs and manufactures reverse vending machines for recycling. |\n\nFeel free to open a PR to add your organization to the list.\n\n## Examples\n\nSee also [runnable examples](https://github.com/kagkarlsson/db-scheduler/tree/master/examples/features/src/main/java/com/github/kagkarlsson/examples).\n\n### Recurring task (_static_)\n\nDefine a _recurring_ task and schedule the task's first execution on start-up using the `startTasks` builder-method. Upon completion, the task will be re-scheduled according to the defined schedule (see [pre-defined schedule-types](#schedules)).\n\n```java\nRecurringTask\u003cVoid\u003e hourlyTask = Tasks.recurring(\"my-hourly-task\", FixedDelay.ofHours(1))\n        .execute((inst, ctx) -\u003e {\n            System.out.println(\"Executed!\");\n        });\n\nfinal Scheduler scheduler = Scheduler\n        .create(dataSource)\n        .startTasks(hourlyTask)\n        .registerShutdownHook()\n        .build();\n\n// hourlyTask is automatically scheduled on startup if not already started (i.e. exists in the db)\nscheduler.start();\n```\n\nFor recurring tasks with multiple instances and schedules, see example [RecurringTaskWithPersistentScheduleMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/RecurringTaskWithPersistentScheduleMain.java).\n\n###  One-time task\n\nAn instance of a _one-time_ task has a single execution-time some time in the future (i.e. non-recurring). The instance-id must be unique within this task, and may be used to encode some metadata (e.g. an id). For more complex state, custom serializable java objects are supported (as used in the example).\n\nDefine a _one-time_ task and start the scheduler:\n\n```java\nTaskDescriptor\u003cMyTaskData\u003e MY_TASK =\n    TaskDescriptor.of(\"my-onetime-task\", MyTaskData.class);\n\nOneTimeTask\u003cMyTaskData\u003e myTaskImplementation =\n    Tasks.oneTime(MY_TASK)\n        .execute((inst, ctx) -\u003e {\n              System.out.println(\"Executed! Custom data, Id: \" + inst.getData().id);\n        });\n\nfinal Scheduler scheduler = Scheduler\n    .create(dataSource, myTaskImplementation)\n    .registerShutdownHook()\n    .build();\n\nscheduler.start();\n```\n\n... and then at some point (at runtime), an execution is scheduled using the `SchedulerClient`:\n\n```java\n// Schedule the task for execution a certain time in the future and optionally provide custom data for the execution\nscheduler.schedule(\n    MY_TASK\n        .instanceWithId(\"1045\")\n        .data(new MyTaskData(1001L))\n        .scheduledTo(Instant.now().plusSeconds(5)));\n```\n\n... or schedule in batches using:\n\n```java\nStream\u003cTaskInstance\u003c?\u003e\u003e taskInstances = Stream.of(\n    MY_TASK.instance(\"my-task-1\", 1),\n    MY_TASK.instance(\"my-task-2\", 2),\n    MY_TASK.instance(\"my-task-3\", 3));\n\nscheduler.scheduleBatch(taskInstances, Instant.now());\n```\n\n### More examples\n\n#### Plain Java\n\n| Example                                                                                                                                                                        | Description                                                                                                                                                                                                             |\n|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| [EnableImmediateExecutionMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/EnableImmediateExecutionMain.java)                                       | When scheduling executions to run `now()` or earlier, the local `Scheduler` will be hinted about this, and \"wake up\" to go check for new executions earlier than it normally would (as configured by `pollingInterval`. |\n| [MaxRetriesMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/MaxRetriesMain.java)                                                                   | How to set a limit on the number of retries an execution can have.                                                                                                                                                      |\n| [ExponentialBackoffMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/ExponentialBackoffMain.java)                                                   | How to use exponential backoff as retry strategy instead of fixed delay as is default.                                                                                                                                  |\n| [ExponentialBackoffWithMaxRetriesMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/ExponentialBackoffWithMaxRetriesMain.java)                       | How to use exponential backoff as retry strategy **and** a hard limit on the maximum number of retries.                                                                                                                 |\n| [TrackingProgressRecurringTaskMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/TrackingProgressRecurringTaskMain.java)                             | Recurring jobs may store `task_data` as a way of persisting state across executions. This example shows how.                                                                                                            |\n| [SpawningOtherTasksMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/SpawningOtherTasksMain.java)                                                   | Demonstrates on task scheduling instances of another by using the `executionContext.getSchedulerClient()`.                                                                                                              |\n| [SchedulerClientMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/SchedulerClientMain.java)                                                         | Demonstrates some of the `SchedulerClient`'s capabilities. Scheduling, fetching scheduled executions etc.                                                                                                                |\n| [RecurringTaskWithPersistentScheduleMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/RecurringTaskWithPersistentScheduleMain.java)                 | Multi-instance recurring jobs where the `Schedule` is stored as part of the `task_data`. For example suitable for multi-tenant applications where each tenent should have a recurring task.                             |\n| [StatefulRecurringTaskWithPersistentScheduleMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/StatefulRecurringTaskWithPersistentScheduleMain.java) |                                                                                                                                                                                                                         |\n| [JsonSerializerMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/JsonSerializerMain.java)                                                           | Overrides serialization of `task_data` from Java-serialization (default) to JSON.                                                                                                                                       |\n| [JobChainingUsingTaskDataMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/JobChainingUsingTaskDataMain.java)                                       | Job chaining, i.e. \"when this instance is done executing, schedule another task.                                                                                                                                        |\n| [JobChainingUsingSeparateTasksMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/JobChainingUsingSeparateTasksMain.java)                             | Job chaining, as above.                                                                                                                                                                                                 |\n| [InterceptorMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/InterceptorMain.java)                                                                 | Using `ExecutionInterceptor` to inject logic before and after execution for all `ExecutionHandler`.                                                                                                                     |\n\n\n\n#### Spring Boot\n\n\n\n| Example                                                                                                                                                         | Description                                                                                                                                                                                                                                |\n|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| [BasicExamples](./examples/spring-boot-example/src/main/java/com/github/kagkarlsson/examples/boot/config/BasicExamplesConfiguration.java)                       | A basic one-time task and recurring task                                                                                                                                                                                                   |\n| [TransactionallyStagedJob](./examples/spring-boot-example/src/main/java/com/github/kagkarlsson/examples/boot/config/TransactionallyStagedJobConfiguration.java) | Example of [transactionally staging a job](https://brandur.org/job-drain), i.e. making sure the background job runs **iff** the transaction commits (along with other db-modifications).                                                   |\n| [LongRunningJob](./examples/spring-boot-example/src/main/java/com/github/kagkarlsson/examples/boot/config/LongRunningJobConfiguration.java)                     | Long-running jobs need to **survive application restarts** and avoid restarting from the beginning. This example demonstrates how to **persisting progress** on shutdown and additionally  a technique for limiting the job to run nightly. |\n| [RecurringStateTracking](./examples/spring-boot-example/src/main/java/com/github/kagkarlsson/examples/boot/config/RecurringStateTrackingConfiguration.java)     | A recurring task with state that can be modified after each run.                                                                                                                                                                           |\n| [ParallelJobSpawner](./examples/spring-boot-example/src/main/java/com/github/kagkarlsson/examples/boot/config/ParallellJobConfiguration.java)                  | Demonstrates how to use a recurring job to spawn one-time jobs, e.g. for parallelization.                                                                                                                                                  |\n| [JobChaining](./examples/spring-boot-example/src/main/java/com/github/kagkarlsson/examples/boot/config/JobChainingConfiguration.java)                           | A one-time job with **multiple steps**. The next step is scheduled after the previous one completes.                                                                                                                                       |\n| [MultiInstanceRecurring](./examples/spring-boot-example/src/main/java/com/github/kagkarlsson/examples/boot/config/MultiInstanceRecurringConfiguration.java)     | Demonstrates how to achieve **multiple recurring jobs** of the same type, but potentially differing schedules and data.                                                                                                                    |\n\n\n\n## Configuration\n\n### Scheduler configuration\n\nThe scheduler is created using the `Scheduler.create(...)` builder. The builder has sensible defaults, but the following options are configurable.\n\n#### Consider tuning\n\n:gear: `.threads(int)`\u003cbr/\u003e\nNumber of threads. Default `10`.\n\n:gear: `.pollingInterval(Duration)`\u003cbr/\u003e\nHow often the scheduler checks the database for due executions. Default `10s`.\u003cbr/\u003e\n\n:gear: `.alwaysPersistTimestampInUTC()`\u003cbr/\u003e\nThe Scheduler assumes that columns for persisting timestamps persist `Instant`s, not `LocalDateTime`s,\n i.e. somehow tie the timestamp to a zone. However, some databases have limited support for such types\n (which has no zone information) or other quirks, making \"always store in UTC\" a better alternative.\nFor such cases, use this setting to always store Instants in UTC.\nPostgreSQL and Oracle-schemas is tested to preserve zone-information. **MySQL** and **MariaDB**-schemas\n_does not_ and should use this setting.\n**NB:** For backwards compatibility, the default behavior\nfor \"unknown\" databases is to assume the database preserves time zone. For \"known\" databases,\nsee the class `AutodetectJdbcCustomization`.\n\n:gear: `.enableImmediateExecution()`\u003cbr/\u003e\nIf this is enabled, the scheduler will attempt to hint to the local `Scheduler` that there are executions to be executed after they are scheduled to\nrun `now()`, or a time in the past. **NB:** If the call to `schedule(..)`/`reschedule(..)` occur from within a transaction, the scheduler might attempt to run\nit before the update is visible (transaction has not committed). It is still persisted though, so even if it is a miss, it will run before the\nnext `polling-interval`. You may also programmatically trigger an early check for due executions using the\nScheduler-method `scheduler.triggerCheckForDueExecutions()`). Default `false`.\n\n:gear: `.registerShutdownHook()`\u003cbr/\u003e\nRegisters a shutdown-hook that will call `Scheduler.stop()` on shutdown. Stop should always be called for a\ngraceful shutdown and to avoid dead executions.\n\n:gear: `.shutdownMaxWait(Duration)`\u003cbr/\u003e\nHow long the scheduler will wait before interrupting executor-service threads. If you find yourself using this,\nconsider if it is possible to instead regularly check `executionContext.getSchedulerState().isShuttingDown()`\nin the ExecutionHandler and abort long-running task. Default `30min`.\n\n:gear: `.enablePriority()`\u003cbr/\u003e\nIt is possible to define a priority for executions which determines the order in which due executions\nare fetched from the database. An execution with a higher value for priority will run before an\nexecution with a lower value (technically, the ordering will be `order by priority desc, execution_time asc`).\nConsider using priorities in the range 0-32000 as the field is defined as a `SMALLINT`. If you need a larger value,\nmodify the schema. For now, this feature is **opt-in**, and column `priority` is only needed by users who choose to\nenable priority via this config setting.\n\nSet the priority per instance using the `TaskInstance.Builder`:\n\n```java\n    scheduler.schedule(\n        MY_TASK\n            .instance(\"1\")\n            .priority(100)\n            .scheduledTo(Instant.now()));\n```\n\nYou can also set the default priority for all tasks of a given type:\n\n```java\nTasks.recurring(\"my-task\", FixedDelay.ofSeconds(5))\n    .defaultPriority(Priority.LOW)\n    .execute(...);\n```\n\n**Note:**\n* When enabling this feature, make sure you have the new necessary indexes defined. If you\nregularly have a state with large amounts of executions both due and future, it might be beneficial\nto add an index on `(execution_time asc, priority desc)` (replacing the old `execution_time asc`).\n* This feature is not recommended for users of **MySQL** and **MariaDB** below version 8.x,\nas they do not support descending indexes.\n* Value `null` for priority may be interpreted differently depending on database (low or high).\n\n#### Polling strategy\n\nIf you are running \u003e1000 executions/s you might want to use the `lock-and-fetch` polling-strategy for lower overhead\n and higher throughput ([read more](#polling-strategy-lock-and-fetch)). If not, the default `fetch-and-lock-on-execute` will be fine.\n\n:gear: `.pollUsingFetchAndLockOnExecute(double, double)`\u003cbr/\u003e\nUse default polling strategy `fetch-and-lock-on-execute`.\u003cbr/\u003e\nIf the last fetch from the database was a full batch (`executionsPerBatchFractionOfThreads`), a new fetch will be triggered\nwhen the number of executions left are less than or equal to `lowerLimitFractionOfThreads * nr-of-threads`.\nFetched executions are not locked/picked, so the scheduler will compete with other instances for the lock\nwhen it is executed. Supported by all databases.\n\u003cbr/\u003eDefaults: `0,5, 3.0`\n\n\n:gear: `.pollUsingLockAndFetch(double, double)`\u003cbr/\u003e\nUse polling strategy `lock-and-fetch` which uses `select for update .. skip locked` for less overhead.\u003cbr/\u003e\nIf the last fetch from the database was a full batch, a new fetch will be triggered\nwhen the number of executions left are less than or equal to `lowerLimitFractionOfThreads * nr-of-threads`.\nThe number of executions fetched each time is equal to `(upperLimitFractionOfThreads * nr-of-threads) - nr-executions-left`.\nFetched executions are already locked/picked for this scheduler-instance thus saving one `UPDATE` statement.\n\u003cbr/\u003eFor normal usage, set to for example `0.5, 1.0`.\n\u003cbr/\u003eFor high throughput\n(i.e. keep threads busy), set to for example `1.0, 4.0`. Currently hearbeats are not updated for picked executions\nin queue (applicable if `upperLimitFractionOfThreads \u003e 1.0`). If they stay there for more than\n`4 * heartbeat-interval` (default `20m`), not starting execution, they will be detected as _dead_ and likely be\nunlocked again (determined by `DeadExecutionHandler`).  Currently supported by PostgreSQL, SQL Server, MySQL v8+.\n\n\n#### Less commonly tuned\n\n:gear: `.heartbeatInterval(Duration)`\u003cbr/\u003e\nHow often to update the heartbeat timestamp for running executions. Default `5m`.\n\n:gear: `.missedHeartbeatsLimit(int)`\u003cbr/\u003e\nHow many heartbeats may be missed before the execution is considered dead. Default `6`.\n\n:gear: `.addExecutionInterceptor(ExecutionInterceptor)`\u003cbr/\u003e\nAdds an `ExecutionInterceptor` which may inject logic around executions. For Spring Boot, simply register a Bean of type `ExecutionInterceptor`.\n\n:gear: `.addSchedulerListener(SchedulerListener)`\u003cbr/\u003e\nAdds an `SchedulerListener` which will receive Scheduler- and Execution-related events. For Spring Boot, simply register a Bean of type `SchedulerListener`.\n\n:gear: `.schedulerName(SchedulerName)`\u003cbr/\u003e\nName of this scheduler-instance. The name is stored in the database when an execution is picked by a scheduler.\nDefault `\u003chostname\u003e`.\n\n:gear: `.tableName(String)`\u003cbr/\u003e\nName of the table used to track task-executions. Change name in the table definitions accordingly when creating\nthe table. Default `scheduled_tasks`.\n\n:gear: `.serializer(Serializer)`\u003cbr/\u003e\nSerializer implementation to use when serializing task data. Default to using standard Java serialization,\nbut db-scheduler also bundles a `GsonSerializer` and `JacksonSerializer`. See examples for a [KotlinSerializer](https://github.com/kagkarlsson/db-scheduler/blob/master/examples/features/src/main/java/com/github/kagkarlsson/examples/kotlin/KotlinSerializer.kt).\nSee also additional documentation under [Serializers](#Serializers).\n\n:gear: `.executorService(ExecutorService)`\u003cbr/\u003e\nIf specified, use this externally managed executor service to run executions. Ideally the number of threads it\nwill use should still be supplied (for scheduler polling optimizations). Default `null`.\n\n:gear: `.deleteUnresolvedAfter(Duration)`\u003cbr/\u003e\nThe time after which executions with unknown tasks are automatically deleted. These can typically be old recurring\ntasks that are not in use anymore. This is non-zero to prevent accidental removal of tasks through a configuration\nerror (missing known-tasks) and problems during rolling upgrades. Default `14d`.\n\n:gear: `.jdbcCustomization(JdbcCustomization)`\u003cbr/\u003e\ndb-scheduler tries to auto-detect the database used to see if any jdbc-interactions need to be customized. This\nmethod is an escape-hatch to allow for setting `JdbcCustomizations` explicitly. Default auto-detect.\n\n:gear: `.commitWhenAutocommitDisabled(boolean)`\u003cbr/\u003e\nBy default no commit is issued on DataSource Connections. If auto-commit is disabled, it is assumed that\ntransactions are handled by an external transaction-manager. Set this property to `true` to override this\nbehavior and have the Scheduler always issue commits. Default `false`.\n\n:gear: `.failureLogging(Level, boolean)`\u003cbr/\u003e\nConfigures how to log task failures, i.e. `Throwable`s thrown from a task execution handler. Use log level `OFF` to disable\n this kind of logging completely. Default `WARN, true`.\n\n### Task configuration\n\nTasks are created using one of the builder-classes in `Tasks`. The builders have sensible defaults, but the following options can be overridden.\n\n| Option  | Default | Description |\n| ------------- | ---- | ------------- |\n| `.onFailure(FailureHandler)`  | see desc.  | What to do when a `ExecutionHandler` throws an exception. By default, _Recurring tasks_ are rescheduled according to their `Schedule` _one-time tasks_ are retried again in 5m. |\n| `.onDeadExecution(DeadExecutionHandler)`  | `ReviveDeadExecution`  | What to do when a _dead executions_ is detected, i.e. an execution with a stale heartbeat timestamp. By default dead executions are rescheduled to `now()`. |\n| `.initialData(T initialData)`  | `null`  | The data to use the first time a _recurring task_ is scheduled. |\n\n\n### Schedules\n\nThe library contains a number of Schedule-implementations for recurring tasks. See class `Schedules`.\n\n| Schedule  | Description                                                                                                                                                                   |\n| ------------- |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `.daily(LocalTime ...)`  | Runs every day at specified times. Optionally a time zone can be specified.                                                                                                   |\n| `.fixedDelay(Duration)`  | Next execution-time is `Duration` after last completed execution. **Note:** This `Schedule` schedules the initial execution to `Instant.now()` when used in `startTasks(...)` |\n| `.cron(String)`  | Spring-style cron-expression (v5.3+). The pattern `-` is interpreted as a [disabled schedule](#disabled-schedules).                                                           |\n\nAnother option to configure schedules is reading string patterns with `Schedules.parse(String)`.\n\nThe currently available patterns are:\n\n| Pattern  | Description |\n| ------------- | ------------- |\n| `FIXED_DELAY\\|Ns`  | Same as `.fixedDelay(Duration)` with duration set to N seconds. |\n| `DAILY\\|12:30,15:30...(\\|time_zone)`  | Same as `.daily(LocalTime)` with optional time zone (e.g. Europe/Rome, UTC)|\n| `-`  | [Disabled schedule](#disabled-schedules) |\n\nMore details on the time zone formats can be found [here](https://docs.oracle.com/javase/8/docs/api/java/time/ZoneId.html#of-java.lang.String-).\n\n### Disabled schedules\n\nA `Schedule` can be marked as disabled. The scheduler will not schedule the initial executions for tasks with a disabled schedule,\nand it will remove any existing executions for that task.\n\n### Serializers\n\nA task-instance may have some associated data in the field `task_data`. The scheduler uses a `Serializer` to read and write this\ndata to the database. By default, standard Java serialization is used, but a number of options is provided:\n\n* `GsonSerializer`\n* `JacksonSerializer`\n* [KotlinSerializer](https://github.com/kagkarlsson/db-scheduler/blob/master/examples/features/src/main/java/com/github/kagkarlsson/examples/kotlin/KotlinSerializer.kt)\n\nFor Java serialization it is recommended to specify a `serialVersionUID` to be able to evolve the class representing the data. If not specified,\nand the class changes, deserialization will likely fail with a `InvalidClassException`. Should this happen, find and set the current auto-generated\n`serialVersionUID` explicitly. It will then be possible to do non-breaking changes to the class.\n\nIf you need to migrate from Java serialization to a `GsonSerializer`, configure the scheduler to use a `SerializerWithFallbackDeserializers`:\n\n```java\n.serializer(new SerializerWithFallbackDeserializers(new GsonSerializer(), new JavaSerializer()))\n```\n\n\n## Third-party extensions\n\n* [bekk/db-scheduler-ui](https://github.com/bekk/db-scheduler-ui) is admin-ui for the scheduler. It shows scheduled executions and supplies simple admin-operations such as\n  \"rerun failed execution now\" and \"delete execution\".\n* [rocketbase-io/db-scheduler-log](https://github.com/rocketbase-io/db-scheduler-log) is an extension providing a history of executions, including failures and exceptions.\n* [piemjean/db-scheduler-mongo](https://github.com/piemjean/db-scheduler-mongo) is an extension for running db-scheduler with a MongoDB database.\n* [osoykan/db-scheduler-additions](https://github.com/osoykan/db-scheduler-additions) adds MongoDB \u0026 Couchbase support on top of Kotlin and Coroutines. It also provides a Ktor plugin for db-scheduler-ui.\n\n## Spring Boot usage\n\nFor Spring Boot applications, there is a starter `db-scheduler-spring-boot-starter` making the scheduler-wiring very simple. (See [full example project](https://github.com/kagkarlsson/db-scheduler/tree/master/examples/spring-boot-example)).\n\n### Prerequisites\n\n- An existing Spring Boot application\n- A working `DataSource` with schema initialized. (In the example HSQLDB is used and schema is automatically applied.)\n\n### Getting started\n\n1. Add the following Maven dependency\n    ```xml\n    \u003cdependency\u003e\n        \u003cgroupId\u003ecom.github.kagkarlsson\u003c/groupId\u003e\n        \u003cartifactId\u003edb-scheduler-spring-boot-starter\u003c/artifactId\u003e\n        \u003cversion\u003e15.1.1\u003c/version\u003e\n    \u003c/dependency\u003e\n    ```\n   **NOTE**: This includes the db-scheduler dependency itself.\n2. In your configuration, expose your `Task`'s as Spring beans. If they are recurring, they will automatically be picked up and started.\n3. If you want to expose `Scheduler` state into actuator health information you need to enable `db-scheduler` health indicator. [Spring Health Information.](https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-features.html#production-ready-health)\n4. Run the app.\n\n### Configuration options\n\nConfiguration is mainly done via `application.properties`. Configuration of scheduler-name, serializer and executor-service is done by adding a bean of type `DbSchedulerCustomizer` to your Spring context.\n\n```\n# application.properties example showing default values\n\ndb-scheduler.enabled=true\ndb-scheduler.heartbeat-interval=5m\ndb-scheduler.polling-interval=10s\ndb-scheduler.polling-strategy=fetch\ndb-scheduler.polling-strategy-lower-limit-fraction-of-threads=0.5\ndb-scheduler.polling-strategy-upper-limit-fraction-of-threads=3.0\ndb-scheduler.table-name=scheduled_tasks\ndb-scheduler.immediate-execution-enabled=false\ndb-scheduler.scheduler-name=\ndb-scheduler.threads=10\ndb-scheduler.priority-enabled=false\n\n# Ignored if a custom DbSchedulerStarter bean is defined\ndb-scheduler.delay-startup-until-context-ready=false\n\ndb-scheduler.polling-strategy=fetch\ndb-scheduler.polling-strategy-lower-limit-fraction-of-threads=0.5\ndb-scheduler.polling-strategy-upper-limit-fraction-of-threads=3.0\n\ndb-scheduler.shutdown-max-wait=30m\n```\n\n## Interacting with scheduled executions using the SchedulerClient\n\nIt is possible to use the `Scheduler` to interact with the persisted future executions. For situations where a full\n`Scheduler`-instance is not needed, a simpler [SchedulerClient](./db-scheduler/src/main/java/com/github/kagkarlsson/scheduler/SchedulerClient.java)\ncan be created using its builder:\n\n```java\nSchedulerClient.Builder.create(dataSource, taskDefinitions).build()\n```\n\nIt will allow for operations such as:\n\n* List scheduled executions\n* Reschedule a specific execution\n* Remove an old executions that have been retrying for too long\n* ...\n\n\n## How it works\n\nA single database table is used to track future task-executions. When a task-execution is due, db-scheduler picks it and executes it. When the execution is done, the `Task` is consulted to see what should be done. For example, a `RecurringTask` is typically rescheduled in the future based on its `Schedule`.\n\nThe scheduler uses optimistic locking or select-for-update (depending on polling strategy) to guarantee that one and only one scheduler-instance gets to pick and run a task-execution.\n\n\n### Recurring tasks\n\nThe term _recurring task_ is used for tasks that should be run regularly, according to some schedule.\n\nWhen the execution of a recurring task has finished, a `Schedule` is consulted to determine what the next time for\nexecution should be, and a future task-execution is created for that time (i.e. it is _rescheduled_).\nThe time chosen will be the nearest time according to the `Schedule`, but still in the future.\n\nThere are two types of recurring tasks, the regular _static_ recurring task, where the `Schedule` is defined statically in the code, and\nthe _dynamic_ recurring tasks, where the `Schedule` is defined at runtime and persisted in the database (still requiring only a single table).\n\n#### Static recurring task\n\nThe _static_ recurring task is the most common one and suitable for regular background jobs since the scheduler automatically schedules\nan instance of the task if it is not present and also updates the next execution-time if the `Schedule` is updated.\n\nTo create the initial execution for a static recurring task, the scheduler has a method `startTasks(...)` that takes a list of tasks\nthat should be \"started\" if they do not already have an existing execution. The initial execution-time is determined by the `Schedule`.\nIf the task already has a future execution (i.e. has been started at least once before), but an updated `Schedule` now indicates another execution-time,\nthe existing execution will be rescheduled to the new execution-time (with the exception of _non-deterministic_ schedules\nsuch as `FixedDelay` where new execution-time is further into the future).\n\nCreate using `Tasks.recurring(..)`.\n\n#### Dynamic recurring task\n\nThe _dynamic_ recurring task is a later addition to db-scheduler and was added to support use-cases where there is need for multiple instances\nof the same type of task (i.e. same implementation) with different schedules. The `Schedule` is persisted in the `task_data` alongside any regular data.\nUnlike the _static_ recurring task, the dynamic one will not automatically schedule instances of the task. It is up to the user to create instances and\nupdate the schedule for existing ones if necessary (using the `SchedulerClient` interface).\nSee the example [RecurringTaskWithPersistentScheduleMain.java](./examples/features/src/main/java/com/github/kagkarlsson/examples/RecurringTaskWithPersistentScheduleMain.java) for more details.\n\nCreate using `Tasks.recurringWithPersistentSchedule(..)`.\n\n### One-time tasks\n\nThe term _one-time task_ is used for tasks that have a single execution-time.\nIn addition to encode data into the `instanceId`of a task-execution, it is possible to store arbitrary binary data in a separate field for use at execution-time. By default, Java serialization is used to marshal/unmarshal the data.\n\nCreate using `Tasks.oneTime(..)`.\n\n### Custom tasks\n\nFor tasks not fitting the above categories, it is possible to fully customize the behavior of the tasks using `Tasks.custom(..)`.\n\nUse-cases might be:\n\n* Tasks that should be either rescheduled or removed based on output from the actual execution\n* ..\n\n\n### Dead executions\n\nDuring execution, the scheduler regularly updates a heartbeat-time for the task-execution.\nIf an execution is marked as executing, but is not receiving updates to the heartbeat-time,\nit will be considered a _dead execution_ after time X. That may for example happen if the\nJVM running the scheduler suddenly exits.\n\nWhen a dead execution is found, the `Task`is consulted to see what should be done. A dead\n`RecurringTask` is typically rescheduled to `now()`.\n\n## Performance\n\nWhile db-scheduler initially was targeted at low-to-medium throughput use-cases, it handles high-throughput use-cases (1000+ executions/second) quite well\ndue to the fact that its data-model is very simple, consisting of a single table of executions.\nTo understand how it will perform, it is useful to consider the SQL statements it runs per batch of executions.\n\n### Polling strategy fetch-and-lock-on-execute\n\nThe original and default polling strategy, `fetch-and-lock-on-execute`, will do the following:\n1. `select` a batch of due executions\n2. For every execution, on execute, try to `update` the execution to `picked=true` for this scheduler-instance. May miss due to competing schedulers.\n3. If execution was picked, when execution is done, `update` or `delete` the record according to handlers.\n\nIn sum per batch: 1 select, 2 * batch-size updates   (excluding misses)\n\n### Polling strategy lock-and-fetch\n\nIn v10, a new polling strategy (`lock-and-fetch`) was added. It utilizes the fact that most databases now have support for `SKIP LOCKED` in `SELECT FOR UPDATE` statements (see [2ndquadrant blog](https://www.2ndquadrant.com/en/blog/what-is-select-skip-locked-for-in-postgresql-9-5/)).\nUsing such a strategy, it is possible to fetch executions pre-locked, and thus getting one statement less:\n\n1. `select for update .. skip locked` a batch of due executions. These will already be picked by the scheduler-instance.\n3. When execution is done, `update` or `delete` the record according to handlers.\n\nIn sum per batch: 1 select-and-update, 1 * batch-size updates   (no misses)\n\n\n### Benchmark test\n\nTo get an idea of what to expect from db-scheduler, see results from the tests run in GCP below.\nTests were run with a few different configurations, but each using 4 competing scheduler-instances running on separate VMs.\nTPS is the approx. transactions per second as shown in GCP.\n\n|                                        | Throughput fetch (ex/s) | TPS fetch (estimates) | Throughput lock-and-fetch (ex/s) | TPS lock-and-fetch (estimates) |\n|----------------------------------------|------------------|--------------------------|---------------------------|-----------------------------------|\n| Postgres 4core 25gb ram, 4xVMs(2-core) |                  |                          |                           |                                   |\n| 20 threads, lower 4.0, upper 20.0      | 2000             | 9000                     | 10600                     | 11500                             |\n| 100 threads, lower 2.0, upper 6.0      | 2560             | 11000                    | 11200                     | 11200                             |\n|                                        |                  |                          |                           |                                   |\n| Postgres 8core 50gb ram, 4xVMs(4-core) |                  |                          |                           |                                   |\n| 50 threads, lower: 0.5, upper: 4.0     | 4000             | 22000                    | 11840                     | 10300                             |\n|                                        |                  |                          |                           |                                   |\n\nObservations for these tests:\n\n* For `fetch-and-lock-on-execute`\n  * TPS ≈ 4-5 * execution-throughput. A bit higher than the best-case 2 * execution-throughput, likely due the inefficiency of missed executions.\n  * throughput did scale with postgres instance-size, from 2000 executions/s on 4core to 4000 executions/s on 8core\n* For `lock-and-fetch`\n  * TPS ≈ 1 * execution-throughput. As expected.\n  * seem to consistently handle 10k executions/s for these configurations\n  * throughput did not scale with postgres instance-size (4-8 core), so bottleneck is somewhere else\n\nCurrently, polling strategy `lock-and-fetch` is implemented only for Postgres. Contributions adding support for more databases are welcome.\n\n### User testimonial\n\nThere are a number of users that are using db-scheduler for high throughput use-cases. See for example:\n\n* https://github.com/kagkarlsson/db-scheduler/issues/209#issuecomment-1026699872\n* https://github.com/kagkarlsson/db-scheduler/issues/190#issuecomment-805867950\n\n### Things to note / gotchas\n\n* There are no guarantees that all instants in a schedule for a `RecurringTask` will be executed. The `Schedule` is consulted after the previous task-execution finishes, and the closest time in the future will be selected for next execution-time. A new type of task may be added in the future to provide such functionality.\n\n* The methods on `SchedulerClient` (`schedule`, `cancel`, `reschedule`) will run using a new `Connection`from the `DataSource` provided. To have the action be a part of a transaction, it must be taken care of by the `DataSource` provided, for example using something like Spring's `TransactionAwareDataSourceProxy`.\n\n* Currently, the precision of db-scheduler is depending on the `pollingInterval` (default 10s) which specifies how often to look in the table for due executions. If you know what you are doing, the scheduler may be instructed at runtime to \"look early\" via `scheduler.triggerCheckForDueExecutions()`. (See also `enableImmediateExecution()` on the `Builder`)\n\n\n## Versions / upgrading\n\nSee [releases](https://github.com/kagkarlsson/db-scheduler/releases) for release-notes.\n\n**Upgrading to 15.x**\n* Priority is a new opt-in feature. To be able to use it, column `priority` and index `priority_execution_time_idx`\n  must be added to the database schema. See table definitions for\n  [postgresql](./db-scheduler/src/test/resources/postgresql_tables.sql),\n  [oracle](./db-scheduler/src/test/resources/oracle_tables.sql) or\n  [mysql](./db-scheduler/src/test/resources/mysql_tables.sql).\n  At some point, this column will be made mandatory. This will be made clear in future release/upgrade-notes.\n\n**Upgrading to 8.x**\n* Custom Schedules must implement a method `boolean isDeterministic()` to indicate whether they will always produce the same instants or not.\n\n**Upgrading to 4.x**\n* Add column `consecutive_failures` to the database schema. See table definitions for [postgresql](./db-scheduler/src/test/resources/postgresql_tables.sql), [oracle](./db-scheduler/src/test/resources/oracle_tables.sql) or [mysql](./db-scheduler/src/test/resources/mysql_tables.sql). `null` is handled as 0, so no need to update existing records.\n\n**Upgrading to 3.x**\n* No schema changes\n* Task creation are preferably done through builders in `Tasks` class\n\n**Upgrading to 2.x**\n* Add column `task_data` to the database schema. See table definitions for [postgresql](./db-scheduler/src/test/resources/postgresql_tables.sql), [oracle](./db-scheduler/src/test/resources/oracle_tables.sql) or [mysql](./db-scheduler/src/test/resources/mysql_tables.sql).\n\n## Building the source\n\n**Prerequisites**\n\n* Java 8+\n* Maven\n\nFollow these steps:\n\n1. Clone the repository.\n   ```\n   git clone https://github.com/kagkarlsson/db-scheduler\n   cd db-scheduler\n   ```\n\n2. Build using Maven (skip tests by adding `-DskipTests=true`)\n   ```\n   mvn package\n   ```\n\n**Recommended spec**\n\nSome users have experienced intermittent test failures when running on a single-core VMs. Therefore, it is recommended to use a minimum of:\n\n- 2 cores\n- 2GB RAM\n\n## FAQ\n\n#### Why `db-scheduler` when there is `Quartz`?\n\nThe goal of `db-scheduler` is to be non-invasive and simple to use, but still solve the persistence problem, and the cluster-coordination problem.\n It was originally targeted at applications with modest database schemas, to which adding 11 tables would feel a bit overkill..\n\n#### Why use a RDBMS for persistence and coordination?\n\nKISS. It's the most common type of shared state applications have.\n\n#### I am missing feature X?\n\nPlease create an issue with the feature request and we can discuss it there.\nIf you are impatient (or feel like contributing), pull requests are most welcome :)\n\n#### Is anybody using it?\n\nYes. It is used in production at a number of companies, and have so far run smoothly.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkagkarlsson%2Fdb-scheduler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkagkarlsson%2Fdb-scheduler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkagkarlsson%2Fdb-scheduler/lists"}