https://github.com/bitfinexcom/caron
Atomic Job enqueuer from Redis lists to popular Job Queues (Sidekiq, Resque, Bull, ...)
https://github.com/bitfinexcom/caron
Last synced: about 1 year ago
JSON representation
Atomic Job enqueuer from Redis lists to popular Job Queues (Sidekiq, Resque, Bull, ...)
- Host: GitHub
- URL: https://github.com/bitfinexcom/caron
- Owner: bitfinexcom
- Created: 2016-06-28T11:56:40.000Z (almost 10 years ago)
- Default Branch: master
- Last Pushed: 2024-04-03T19:00:21.000Z (about 2 years ago)
- Last Synced: 2025-04-11T06:13:50.409Z (about 1 year ago)
- Language: JavaScript
- Homepage: https://bitfinex.com
- Size: 63.5 KB
- Stars: 10
- Watchers: 4
- Forks: 9
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# caron
Atomic Job enqueuer from Redis lists to popular Job Queues (Sidekiq, Bull, ...)
**Caron** pops messages from a redis list and atomically creates a Job for the specified Job Queue.
Uses `lua` scripting internally to provide atomicity (http://redis.io/commands/EVAL)
## Support
* [Sidekiq-5.0.4](https://github.com/mperham/sidekiq)
* [Bull-3.2.0](https://github.com/OptimalBits/bull)
## Install
```
npm install -g caron
```
## Usage
```
$ caron --help
Usage: caron [options]
Options:
-h, --help output usage information
-V, --version output the version number
-t, --type queue type [sidekiq | bull]
-l, --list source redis list (i.e: global_jobs)
-r, --redis redis url (i.e: redis://127.0.0.1:6379)
-f, --freq poll frequency (in milliseconds) - default: 10
-b, --batch max number of jobs created per batch - default: 1000
--q_prefix redis queue prefix (i.e: "bull")
--def_queue default dest queue - default: default
--def_worker default Job Queue worker - default: BaseJob
--def_attempts default Bull Job attempts - default: 1
--q_lifo Bull LIFO mode
```
```
caron --type sidekiq --list sidekiq_jobs --redis "redis://127.0.0.1:6379" --freq 25
```
Debug mode:
```
DEBUG=caron:* caron -t bull -l bull_test
```
### Examples
##### Push from redis-cli
```bash
// Sidekiq job enqueue
redis-cli > lpush "sidekiq_jobs" "{\"$queue\":\"critical\",\"$class\":\"BackendJob\",\"foo\":\"bar\",\"my\":\"stuff\",\"other\":\"stuff\",\"other\":{\"f\":5}}"
// Bull job enqueue
redis-cli > lpush "bull_jobs" "{\"$queue\":\"critical\",\"$attempts\":4,\"$backoff\":{\"type\":\"exponential\",\"delay\":5000}, \"foo\":\"bar\",\"my\":\"stuff\",\"other\":{\"f\":5}}"
```
##### Push from Node.js
```js
'use strict'
const Redis = require('ioredis')
var redis = new Redis()
redis.lpush('sidekiq_test', JSON.stringify({ foo: 'bar', '$queue': 'critical', '$class': 'MyCriticalJob', other: { a: 1, b: 2 } }))
redis.lpush('bull_test', JSON.stringify({ foo: 'bar', '$queue': 'priority', '$attempts': 5, '$backoff': { type: 'exponential', delay: 5000 }, other: { a: 1, b: 2 } }))
redis.lpush('bull_test', JSON.stringify({ foo: 'bar', '$queue': 'lazy', '$attempts': 1, '$delay': 5000, other: { a: 1, b: 2 } }))
```
##### Push from Ruby
```ruby
require 'redis'
require 'json'
rcli = Redis.new
rcli.lpush('sidekiq_test', JSON.dump({ 'foo' => 'bar', '$queue' => 'critical', '$class' => 'MyCriticalJob', 'other' => { 'a' => 1, 'b' => 2 } }))
rcli.lpush('bull_test', JSON.dump({ 'foo' => 'bar', '$queue' => 'priority', '$attempts' => 5, '$backoff' => { 'type' => exponential', 'delay' => 5000 }, 'other' => { 'a' => 1, 'b' => 2 } }))
rcli.lpush('bull_test', JSON.dump({ 'foo' => 'bar', '$queue' => 'lazy', '$attempts' => 1, '$delay' => 5000, other => { 'a' => 1, 'b' => 2 } }))
rcli.lpush('bull_test', JSON.dump({ 'foo' => 'bar', '$queue' => 'lazy', '$attempts' => 1, '$delay' => 5000, other => { 'a' => 1, 'b' => 2 } }))
```