https://github.com/umbrellio/simple_mutex
Redis-based mutex library for using with Sidekiq jobs and batches
https://github.com/umbrellio/simple_mutex
Last synced: about 1 year ago
JSON representation
Redis-based mutex library for using with Sidekiq jobs and batches
- Host: GitHub
- URL: https://github.com/umbrellio/simple_mutex
- Owner: umbrellio
- License: mit
- Created: 2021-10-22T08:29:31.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2023-10-11T08:17:50.000Z (over 2 years ago)
- Last Synced: 2024-10-14T02:18:29.219Z (over 1 year ago)
- Language: Ruby
- Homepage:
- Size: 36.1 KB
- Stars: 3
- Watchers: 6
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# SimpleMutex · [](https://badge.fury.io/rb/simple_mutex) [](https://coveralls.io/github/umbrellio/simple_mutex?branch=main)
`SimpleMutex::Mutex` - Redis-based locks with ability to store custom data inside them.
`SimpleMutex::SidekiqSupport::JobWrapper` - wrapper for Sidekiq jobs that generates locks using
job's class name and arguments (optional)
`SimpleMutex:SidekiqSupport::JobMixin` - mixin for Sidekiq jobs with DSL simplifying usage
of `SimpleMutex::SidekiqSupport::JobWrapper`
`SimpleMutex::SidekiqSupport::JobCleaner` - cleaner for leftover locks created by SimpleMutex::Job
if Sidekiq dies unexpectedly.
`SimpleMutex::SidekiqSupport::Batch` - wrapper for Sidekiq Pro batches that use SimpleMutex::Mutex
to prevent running multiple batch instances.
`SimpleMutex:SidekiqSupport::BatchCleaner` - cleaner for leftover lock created by SimpleMutex::Batch
if Sidekiq dies unexpectedly.
`SimpleMutex::Helper` - auxiliary class for debugging purposes. Allows to inspect existing locks.
## Configuration
Providing Redis instance before using gem is mandatory.
```ruby
SimpleMutex.redis = Redis.new(
# ...
)
```
Providing logger is optional (used by `SimpleMutex::SidekiqSupport::JobCleaner` and
`SimpleMutex::SidekiqSupport::BatchCleaner`).
```ruby
SimpleMutex.logger = Logger.new(
# ...
)
```
When using gem with Ruby on Rails you can set those in initializers
## SimpleMutex::Mutex Usage
### Initialization
#### Arguments
##### mandatory
* `lock_key` - string that identifies lock, mandatory. Two pieces of locked code can't be run
simultaneously if they use same `lock_key`. They don't interfere with each other if different
`lock_key`'s are used
##### optional
Keyword arguments are used for optional args.
* `expires_in:` - mutex TTL in second (or ActiveSupport::Numeric time interval), lock will be
removed by redis automatically when expired, lock will expire in 1 hour (`3600`) if not provided
* `signature:` - string used to determine ownership of lock, checked when manually deleting lock,
will be generated by `SecureRandom.uuid` if not provided
* `payload:` - any object that can be serialized as JSON, `nil` if not provided
#### Example
```ruby
SimpleMutex::Mutex
.new(
"some_lock_key",
expires_in: 3600,
signature: "qwe123",
payload: { "started_at" => Time.now }
)
```
### Wrapping block in mutex
You can use method `#with_lock` to wrap code block in mutex
```ruby
SimpleMutex::Mutex
.new(
"some_lock_key",
expires_in: 3600,
signature: "qwe123",
payload: { "started_at" => Time.now }
).with_lock do
# your code
end
```
Method has delegator defined on class, so it can be used without manual instantiation
```ruby
SimpleMutex::Mutex
.with_lock(
"some_lock_key",
expires_in: 3600,
signature: "qwe123",
payload: { "started_at" => Time.now }
) do
# your code
end
```
### Manual lock control
##### Using mutex instance
```ruby
mutex = SimpleMutex::Mutex.new(
"some_lock_key",
expires_in: 3600,
signature: "qwe123",
payload: { "started_at" => Time.now }
)
mutex.lock!
# your code
mutex.unlock!
```
If you for some reason don't want exceptions to be raised when obtaining/deleting lock is failed,
you can use non-! methods.
```ruby
mutex = SimpleMutex::Mutex.new("some_lock_key")
# obtaining of lock is not guaranteed
mutex.lock
# but you can check if it is obtained (true if lock with correct signature exists)
mutex.lock_obtained?
# releasing of lock is not guaranteed
mutex.unlock
```
##### Using without instance
There are `::lock`/`::lock!`/`::unlock`/`::unlock!` methods defined on class if you don't want to
explicitly use initializer (though it still will be used behind the scenes as `::lock` and `::lock!`
class methods are just delegators).
Mutexes have random `signature` stored inside to determine ownership. By default it prevents
deleting locks with signature different from provided. You can use `force: true` to ignore
signature check.
`::lock` and `::lock!` class methods accept same arguments as in `::new`
`::unlock` and `::unlock!` accept next arguments:
* `lock_key` - same as in `::new`
* `signature:` - same as in `::new`
* `force:` - boolean, signature will be ignored if `true`, optional, `false` by default
```ruby
SimpleMutex::Mutex.lock!("some_lock_key", signature: "abra_kadabra")
# This will work because signature is same as in lock
SimpleMutex::Mutex.unlock!("some_lock_key", signature: "abra_kadabra")
# This won't work, because signature is missing
SimpleMutex::Mutex.unlock!("some_lock_key")
# This won't work, because signature is different
SimpleMutex::Mutex.unlock!("some_lock_key", signature: "alakazam")
# This will work because of force: true
SimpleMutex::Mutex.unlock!("some_lock_key", force: true)
# This will work because of force: true
SimpleMutex::Mutex.unlock!("some_lock_key", signature: "alakazam", force: true)
```
### Getting signature from instance
You can get signature from instance if you want. By default it is UUID generated by SecureRandom.
```ruby
mutex = SimpleMutex::Mutex.new("some_lock_key")
mutex.signature
```
## SimpleMutex::SidekiqSupport::JobWrapper Usage
This class made to simplify usage for locking of sidekiq jobs. It will create lock with
`lock_key` based on job's `class.name` and it's arguments if `lock_with_params: true`.
Job's ID (`jid`) and time when job's execution is started will be stored inside mutex value.
```ruby
class SomeJob
include Sidekiq::Worker
def perform(*args)
SimpleMutex::SidekiqSupport::JobWrapper.new(
self,
params: args,
lock_with_params: true,
expires_in: 1.hour,
payload: { this_is_optional: true }
).with_redlock do
# your code
end
end
end
```
`params` will be used to generate `lock_key` if `lock_with_params: true`.
`expires_in:` is in seconds, optional, 5 hours by default.
`payload:` optional serializable object.
## SimpleMutex::SidekiqSupport::Batch Usage
This is wrapper for `Sidekiq::Batch` (from Sidekiq Pro) that helps to prevent running two
similar batches.
```ruby
batch = SimpleMutex::SidekiqSupport::Batch.new(
lock_key: "my_batch",
expires_in: 23.hours.to_i,
)
batch.description = "batch of MyJobs"
batch.on(:success, self.class, {}) # you can add custom callbacks like with Sidekiq::Batch
batch.on(:death , self.class, {})
batch.jobs do
set_of_job_attributes.each do |job_attributes|
MyJob.perform(job_attributes)
end
end
```
* `lock_key` - manatory lock key
* `expires_in:` - optional TTL, 6 hours if not provided
## SimpleMutex::SidekiqSupport::JobCleaner Usage
If you use SimpleMutex for locking jobs via `SimpleMutex::SidekiqSupport::Job`, when Sidekiq dies
unexpectedely, there can be leftover mutexes for dead jobs. To delete them you can use:
```ruby
SimpleMutex::SidekiqSupport::JobCleaner.unlock_dead_jobs
```
## SimpleMutex::SidekiqSupport::BatchCleaner Usage
If you use SimpleMutex for locking Batches via `SimpleMutex::SidekiqSupport::Batch`, when Sidekiq
dies unexpectedely, there can be leftover mutexes for dead batches. To delete them you can use:
```ruby
SimpleMutex::SidekiqSupport::BatchCleaner.unlock_dead_batches
```
## SimpleMutex::Helper Usage
Getting lock by `lock_key` (returns nil if no such lock)
```ruby
SimpleMutex::Helper.get("some_lock_key")
```
Listing existing locks.
```ruby
SimpleMutex::Helper.list(mode: :default)
```
`mode:` paramater allows to filter locks by type:
* `:all` - all locks including manual
* `:job` - job locks
* `:batch` - batch locks
* `:default` - job and batch locks
## SimpleMutex::SidekiqSupport::JobMixin Usage
Base Job class
```ruby
class ApplicationJob
include Sidekiq::Worker
include SimpleMutex::SidekiqSupport::JobMixin
class << self
def inherited(job_class)
# Setting default timeout for mutex.
job_class.set_job_timeout(5 * 60 * 60) # 5 hours
job_class.prepend(
Module.new do
def perform(*args)
with_redlock(args) { super }
end
end,
)
end
end
end
```
DSL:
- `locking!` - enables locking with simple_mutex for jobs of this class
- `lock_with_params!` - locks are specific for set of arguments. Same job with other arguments
can still be called.
- `skip_locking_error?` - suppresses `SimpleMutex::Mutex::LockError`
- `set_job_timeout` - redis mutex TTL in seconds (will be removed by redis itself on timeout)
Example:
```ruby
class SpecificJob < ApplicaionJob
locking!
lock_with_params!
set_job_timeout 6 * 60 * 60
def perform
# ...
end
end
```
You can also override error processing for SimpleMutex::Mutex::LockError
```ruby
# DEFAULT ERROR PROCESSING
# def process_locking_error(error)
# raise error unless self.class.skip_locking_error?
# end
class SpecificJob < ApplicaionJob
locking!
def perform
# ...
end
def process_locking_error(error)
SomeLogger.error(error.msg)
raise error unless self.class.skip_locking_error?
end
end
```
## Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/umbrellio/simple_mutex.
## License
Released under MIT License.
## Authors
Team Umbrellio
---
