Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/joker1007/crono_trigger
Simple Asynchronous Job Scheduler for Rails.
https://github.com/joker1007/crono_trigger
Last synced: 2 days ago
JSON representation
Simple Asynchronous Job Scheduler for Rails.
- Host: GitHub
- URL: https://github.com/joker1007/crono_trigger
- Owner: joker1007
- License: mit
- Created: 2017-06-18T19:16:14.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2024-10-25T06:43:26.000Z (14 days ago)
- Last Synced: 2024-10-26T17:33:44.162Z (13 days ago)
- Language: Ruby
- Homepage:
- Size: 4.41 MB
- Stars: 47
- Watchers: 6
- Forks: 16
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# CronoTrigger
[](https://badge.fury.io/rb/crono_trigger)
[](https://codecov.io/gh/joker1007/crono_trigger)
Asynchronous Job Scheduler for Rails.
The purpose of this gem is to integrate job schedule into Service Domain.
Because of it, this gem uses ActiveRecord model as definition of job schedule.
## Installation
Add this line to your application's Gemfile:
```ruby
gem 'crono_trigger'
```
And then execute:
$ bundle
Or install it yourself as:
$ gem install crono_trigger
## Breaking Changes
### Update from v0.6.x
In previous version, now is `2023-06-07T18:00:00+00:00`, a cron definition is `0 1 * * * *`, and `started_at` is `2023-06-08T1:00:00+00:00`.
In this case, `next_execute_at` was `2023-06-09T1:00:00+00:00`
From v0.7.0, if the cron definition and `started_at` match, include the time of `started_at` as the `next_execute_at`.
For example, now is `2023-06-07T18:00:00+00:00`, a cron definition is `0 1 * * * *`, and `started_at` is `2023-06-08T1:00:00+00:00`.
In this case, `next_execute_at` is `2023-06-08T1:00:00+00:00`.
If the current time is past `started_at`, the `next_execute_at` is based on the current time.
### Update from v0.3.x
#### Create crono_trigger system tables
```
$ rails g crono_trigger:install # => create migrations
$ rake db:migrate
```
#### Add `locked_by:string` column to CronoTrigger::Schedulable model
```
$ rails g migration add_locked_by_column_to_your_model
$ rake db:migrate
```
```ruby
class AddLockedByColumnToYourModel < ActiveRecord::Migration[5.2]
def change
add_column :your_models, :locked_by, :string
end
end
```
## Usage
#### Execute `crono_trigger:model` generator.
```
$ rails g crono_trigger:model mail_notification
create db/migrate/20170619064928_create_mail_notifications.rb
create app/models/mail_notification.rb
# ...
```
#### Migration sample
```ruby
class CreateMailNotifications < ActiveRecord::Migration
def change
create_table :mail_notifications do |t|
# columns for CronoTrigger::Schedulable
t.string :cron
t.datetime :next_execute_at
t.datetime :last_executed_at
t.integer :execute_lock, limit: 8, default: 0, null: false
t.datetime :started_at
t.datetime :finished_at
t.string :last_error_name
t.string :last_error_reason
t.datetime :last_error_time
t.integer :retry_count, default: 0, null: false
t.timestamps
end
add_index :mail_notifications, [:next_execute_at, :execute_lock, :started_at, :finished_at], name: "crono_trigger_index_on_mail_notifications"
end
end
```
#### Implement `#execute` method
```ruby
class MailNotification < ActiveRecord::Base
include CronoTrigger::Schedulable
self.crono_trigger_options = {
retry_limit: 5,
retry_interval: 10,
exponential_backoff: true,
execute_lock_timeout: 300,
}
# `execute`, `retry` callback is defined
# can use `before_execute`, `after_execute`, `around_execute`
# `before_retry`, `after_retry`, `around_retry`
# If execute method raise Exception, worker retry task until reach `retry_limit`
# If `retry_count` reaches `retry_limit`, task schedule is reset.
#
# If record has cron value, reset process set next execution time by cron definition
# If record has no cron value, reset process clear next execution time
def execute
send_mail
throw :retry # break execution and retry task
throw :abort # break execution
throw :ok # break execution and handle task as success
throw :ok_without_reset # break execution and handle task as success but without schedule reseting and unlocking
end
end
# one time schedule
MailNotification.create.activate_schedule!(at: Time.current.since(5.minutes))
# cron schedule
MailNotification.create(cron: "0 12 * * *").activate_schedule!
# or
MailNotification.new(cron: "0 12 * * *").activate_schedule!.save
# if update cron column or timezone column
# update next_execute_at automatically by before_update callback
mail = MailNotification.create(cron: "0 12 * * *").activate_schedule!
mail.next_execute_at # => next 12:00 with Time.zone
mail.update(cron: "0 13 * * *")
mail.next_execute_at # => next 13:00 with Time.zone
mail.update(timezone: "Asia/Tokyo")
mail.next_execute_at # => next 13:00 with Asia/Japan
```
#### Run Worker
use `crono_trigger` command.
`crono_trigger` command accepts model class names.
For example,
```
$ crono_trigger MailNotification
```
And other options is following.
```
$ crono_trigger --help
Usage: crono_trigger [options] MODEL [MODEL..]
-f, --config-file=CONFIG Config file (ex. ./crono_trigger.rb)
-e, --environment=ENV Set environment name (ex. development, production)
-p, --polling-thread=SIZE Polling thread size (Default: 1)
-i, --polling-interval=SECOND Polling interval seconds (Default: 5)
-c, --concurrency=SIZE Execute thread size (Default: 25)
-r, --fetch-records=SIZE Record count fetched by polling thread (Default: concurrency * 3)
-l, --log=LOGFILE Set log output destination (Default: STDOUT or ./crono_trigger.log if daemonize is true)
--log-level=LEVEL Set log level (Default: info)
-d, --daemonize Daemon mode
--pid=PIDFILE Set pid file
-h, --help Prints this help
```
#### Handle errors
This gem provides the following options to handle errors:
```ruby
CronoTrigger.configure do |config|
# These handlers are called when the execute method fails even after retries.
config.error_handlers << proc do |exception, record|
ActiveRecord::Base.logger.error("Failed to process #{record.class}##{record.id}: #{exception} (#{exception.class})")
end
# These handlers are called when an exception occurs outside of the execute method.
config.global_error_handlers << proc do |exception|
ActiveRecord::Base.logger.error("#{exception} (#{exception.class})\n#{exception.backtrace.join("\n")}")
end
# db_error_retriable_options are passed to `Retriable.retriable` used in the idempotent code that accesses the database
# except for the code processing each record.
# Here is the default value.
config.db_error_retriable_options = {
on: {
ActiveRecord::ConnectionNotEstablished => nil,
},
}
end
```
For example, if you would like to reconnect to the database before retry for some reason, you can do so using `on_retry` option as follows:
```ruby
CronoTrigger.configure do |config|
config.db_error_retriable_options = {
on: {
ActiveRecord::ConnectionNotEstablished => nil,
Mysql2::Error::ConnectionError => nil,
ActiveRecord::StatementInvalid => /MySQL server is running with the --read-only option/,
},
on_retry: proc do |exception, try, elapsed_time, interval|
ActiveRecord::Base.logger.info("#{try}th: Retry on #{exception.class}: #{exception}")
next unless exception.is_a?(ActiveRecord::StatementInvalid)
ActiveRecord::Base.logger.info("#{try}th: Reconnect to MySQL on #{exception.class}: #{exception}")
# NOTE: The connection acquired here might be different from the one used in the code that raised the error,
# but we don't have a way to get the latter connection.
ActiveRecord::Base.connection_pool.with_connection(&:reconnect!)
end,
}
end
```
## Specification
### Columns
| name | type | required | rename | description |
| ----------------- | -------- | -------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| cron | string | no | no | Recurring schedule formatted by cron style |
| next_execute_at | datetime | yes | yes | Timestamp of next execution. Worker executes task if this column <= now |
| last_executed_at | datetime | no | yes | Timestamp of last execution |
| timezone | datetime | no | yes | Timezone name (Parsed by tzinfo) |
| execute_lock | integer | yes | yes | Timestamp of fetching record in order to hide record from other transaction during execute lock timeout.
when execution complete this column is reset to 0 |
| started_at | datetime | no | yes | Timestamp of schedule activated |
| finished_at | datetime | no | yes | Timestamp of schedule deactivated |
| last_error_name | string | no | no | Class name of last error |
| last_error_reason | string | no | no | Error message of last error |
| last_error_time | datetime | no | no | Timestamp of last error occured |
| retry_count | integer | no | no | Retry count.
If execution succeed retry_count is reset to 0 |
| current_cycle_id | string | no | yes | UUID that is updated when the schedule is resetted successfully |
You can rename some columns.
ex. `crono_trigger_options[:next_execute_at_column_name] = "next_time"`
## Admin Web
### Standalone mode
```
$ crono_trigger-web --rails
```
### Mount as Rack app
```ruby
# config/routes.rb
require "crono_trigger/web"
mount CronoTrigger::Web => '/crono_trigger'
```
## Rollbar integration
This gem has rollbar plugin.
If `crono_trigger/rollbar` is required, Add Rollbar logging process to `CronoTrigger.config.error_handlers`
## Active Support Instrumentation Events
This gem provides the following events for [Active Support Instrumentation](https://guides.rubyonrails.org/active_support_instrumentation.html).
### monitor.crono\_trigger
This event is triggered every 20 seconds by the first active worker in worker_id order, so note that other workers don't receive the event.
| Key | Value |
| ------------------------ | ----------------------------------------------------------------------------- |
| model\_name | The model name |
| executable\_count | The number of executable records |
| max\_lock\_duration\_sec | The maximum amount of time since locked records started being processed |
| max\_latency\_sec | The maximum amount of time since executable records got ready to be processed |
### process\_record.crono\_trigger
This event is triggered every time a record finishes being processed.
| Key | Value |
| ------- | -------------------- |
| record | The processed record |
## Development
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
## Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/joker1007/crono_trigger.
## License
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).