Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/sonus21/rqueue
Rqueue aka Redis Queue [Task Queue, Message Broker] for Spring framework
https://github.com/sonus21/rqueue
asynchronous-tasks delayed-jobs delayed-queue java queue queue-tasks queue-workers queued-jobs redis rqueue scheduled-jobs scheduled-tasks scheduler spring spring-boot spring-framework task-executor task-manager task-scheduler workers
Last synced: about 5 hours ago
JSON representation
Rqueue aka Redis Queue [Task Queue, Message Broker] for Spring framework
- Host: GitHub
- URL: https://github.com/sonus21/rqueue
- Owner: sonus21
- License: apache-2.0
- Created: 2019-10-18T10:36:21.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2024-09-19T16:12:13.000Z (9 days ago)
- Last Synced: 2024-09-24T22:33:30.917Z (3 days ago)
- Topics: asynchronous-tasks, delayed-jobs, delayed-queue, java, queue, queue-tasks, queue-workers, queued-jobs, redis, rqueue, scheduled-jobs, scheduled-tasks, scheduler, spring, spring-boot, spring-framework, task-executor, task-manager, task-scheduler, workers
- Language: Java
- Homepage: https://sonus21.github.io/rqueue
- Size: 9.65 MB
- Stars: 337
- Watchers: 14
- Forks: 57
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md
Awesome Lists containing this project
README
Rqueue: Redis Queue, Task Queue, Scheduled Queue for Spring and Spring Boot
[](https://circleci.com/gh/sonus21/rqueue/tree/master)
[](https://coveralls.io/github/sonus21/rqueue?branch=master)
[](https://repo1.maven.org/maven2/com/github/sonus21/rqueue-core)
[](https://javadoc.io/doc/com.github.sonus21/rqueue-core)
[](LICENSE)
**Rqueue** is an asynchronous task executor(worker) built for spring and spring-boot framework based
on the spring framework's messaging library backed by Redis. It can be used as message broker as
well, where all services code is in Spring.
## Features
* **Instant delivery** : Instant execute this message in the background
* **Message scheduling** : A message can be scheduled for any arbitrary period
* **Unique message** : Unique message processing for a queue based on the message id
* **Periodic message** : Process same message at certain interval
* **Priority tasks** : task having some special priority like high, low, medium
* **Message delivery** : It's guaranteed that a message is consumed **at least once**. (Message
would be consumed by a worker more than once due to the failure in the underlying
worker/restart-process etc, otherwise exactly one delivery)
* **Message retry** : Message would be retried automatically on application crash/failure/restart
etc.
* **Automatic message serialization and deserialization**
* **Message Multicasting** : Call multiple message listeners on every message
* **Batch Message Polling** : Fetch multiple messages from Redis at once
* **Metrics** : In flight messages, waiting for consumption and scheduled messages
* **Competing Consumers** : multiple messages can be consumed in parallel by different
workers/listeners.
* **Concurrency** : Concurrency of any listener can be configured
* **Queue Priority** :
* Group level queue priority(weighted and strict)
* Sub queue priority(weighted and strict)
* **Long execution job** : Long running jobs can check in periodically.
* **Execution Backoff** : Exponential and fixed back off (default fixed back off)
* **Middleware** : Add one or more middleware, middlewares are called before listener method.
* **Callbacks** : Callbacks for dead letter queue, discard etc
* **Events** : 1. Bootstrap event 2. Task execution event.
* **Redis connection** : A different redis setup can be used for Rqueue
* **Redis cluster** : Redis cluster can be used with Lettuce client.
* **Redis Sentinel** : Redis sentinel can be used with Rqueue.
* **Reactive Programming** : Supports reactive Redis and spring webflux
* **Web Dashboard** : Web dashboard to manage a queue and queue insights including latency
### Requirements
* Spring 5+, 6+
* Java 1.8+,17
* Spring boot 2+,3+
* Lettuce client for Redis cluster
* Read master preference for Redis cluster
## Getting Started
### Dependency
Snapshot
Version: [https://s01.oss.sonatype.org/content/repositories/snapshots/com/github/sonus21/](https://s01.oss.sonatype.org/content/repositories/snapshots/com/github/sonus21/)
Release Version: [Maven central](https://search.maven.org/search?q=g:com.github.sonus21)
#### Spring Boot
**NOTE:**
* For spring boot 2.x use Rqueue 2.x
* For spring boot 3.x use Rqueue 3.x
Get the latest one
from [Maven central](https://search.maven.org/search?q=g:com.github.sonus21%20AND%20a:rqueue-spring-boot-starter)
* Add dependency
* Gradle
```groovy
implementation 'com.github.sonus21:rqueue-spring-boot-starter:2.13.1-RELEASE'
```
* Maven
```xml
com.github.sonus21
rqueue-spring-boot-starter
2.13.1-RELEASE
```
No additional configurations are required, only dependency is required.
---
#### Spring Framework
**NOTE**
* For spring framework 5.x use rqueue-spring 2.x
* For spring framework 6.x use rqueue-spring 3.x
Get the latest one
from [Maven central](https://search.maven.org/search?q=g:com.github.sonus21%20AND%20a:rqueue-spring)
* Add Dependency
* Gradle
```groovy
implementation 'com.github.sonus21:rqueue-spring:2.13.1-RELEASE'
```
* Maven
```xml
com.github.sonus21
rqueue-spring
2.13.1-RELEASE
```
* Add annotation `EnableRqueue` on application config class
* Provide a RedisConnectionFactory bean
###### Configuration
```java
@EnableRqueue
public class Application {
@Bean
public RedisConnectionFactory redisConnectionFactory() {
// return a redis connection factory
}
}
```
---
### Message publishing/Task submission
All messages need to be sent using `RqueueMessageEnqueuer` bean's `enqueueXXX`, `enqueueInXXX`
and `enqueueAtXXX` methods. It has handful number of `enqueue`, `enqueueIn`, `enqueueAt` methods, we
can use any one of them based on the use case.
```java
public class MessageService {
@AutoWired
private RqueueMessageEnqueuer rqueueMessageEnqueuer;
public void doSomething() {
rqueueMessageEnqueuer.enqueue("simple-queue", "Rqueue is configured");
}
public void createJOB(Job job) {
rqueueMessageEnqueuer.enqueue("job-queue", job);
}
// send notification in 30 seconds
public void sendNotification(Notification notification) {
rqueueMessageEnqueuer.enqueueIn("notification-queue", notification, 30 * 1000L);
}
// enqueue At example
public void createInvoice(Invoice invoice, Instant instant) {
rqueueMessageEnqueuer.enqueueAt("invoice-queue", invoice, instant);
}
// enqueue with priority, when sub queues are used as explained in the queue priority section.
enum SmsPriority {
CRITICAL("critical"),
HIGH("high"),
MEDIUM("medium"),
LOW("low");
private String value;
}
public void sendSms(Sms sms, SmsPriority priority) {
rqueueMessageEnqueuer.enqueueWithPriority("sms-queue", priority.value(), sms);
}
// Index chat every 1 minute
public void sendPeriodicEmail(Email email) {
rqueueMessageEnqueuer.enqueuePeriodic("chat-indexer", chatIndexer, 60_000);
}
}
```
---
### Worker/Consumer/Task Executor/Listener
Any method that's part of spring bean, can be marked as worker/message listener
using `RqueueListener` annotation
```java
@Component
@Slf4j
public class MessageListener {
@RqueueListener(value = "simple-queue")
public void simpleMessage(String message) {
log.info("simple-queue: {}", message);
}
@RqueueListener(value = "job-queue", numRetries = "3",
deadLetterQueue = "failed-job-queue", concurrency = "5-10")
public void onMessage(Job job) {
log.info("Job alert: {}", job);
}
@RqueueListener(value = "push-notification-queue", numRetries = "3",
deadLetterQueue = "failed-notification-queue")
public void onMessage(Notification notification) {
log.info("Push notification: {}", notification);
}
@RqueueListener(value = "sms", priority = "critical=10,high=8,medium=4,low=1")
public void onMessage(Sms sms) {
log.info("Sms : {}", sms);
}
@RqueueListener(value = "chat-indexing", priority = "20", priorityGroup = "chat")
public void onMessage(ChatIndexing chatIndexing) {
log.info("ChatIndexing message: {}", chatIndexing);
}
@RqueueListener(value = "chat-indexing-daily", priority = "10", priorityGroup = "chat")
public void onMessage(ChatIndexing chatIndexing) {
log.info("ChatIndexing message: {}", chatIndexing);
}
// checkin job example
@RqueueListener(value = "chat-indexing-weekly", priority = "5", priorityGroup = "chat")
public void onMessage(ChatIndexing chatIndexing,
@Header(RqueueMessageHeaders.JOB) com.github.sonus21.rqueue.core.Job job) {
log.info("ChatIndexing message: {}", chatIndexing);
job.checkIn("Chat indexing...");
}
}
```
---
## Dashboard
Link: [http://localhost:8080/rqueue](http://localhost:8080/rqueue)
[](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/stats-graph.png)
#### Queue Statistics
Micrometer based dashboard for queue
[](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/grafana-dashboard.png)
#### Message Waiting For Execution
[](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/queue-explore.png)
#### Recent jobs details
[](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/jobs.png)
---
## Status
Rqueue is stable and production ready, it's processing 100K+ messages daily in production
environment.
**Some of the Rqueue Users**
**We would love to add your organization name here, if you're one of the Rqueue users, please raise
a
PR/[issue](https://github.com/sonus21/rqueue/issues/new?template=i-m-using-rqueue.md&title=Add+my+organisation+in+Rqueue+Users)
.**
---
## Support
* Please report bug,question,feature(s)
to [issue](https://github.com/sonus21/rqueue/issues/new/choose) tracker.
* Ask question on StackOverflow using [#rqueue](https://stackoverflow.com/tags/rqueue) tag
## Contribution
You are most welcome for any pull requests for any feature/bug/enhancement. You would need Java8 and
gradle to start with. In root `build.gradle` file comment out spring related versions, or set
environment variables for Spring versions. You can use [module, class and other diagrams](https://sourcespy.com/github/sonus21rqueue/)
to familiarise yourself with the project.
**Please format your code with Google Java formatter.**
## Links
* Documentation: [https://sonus21.github.io/rqueue](https://sonus21.github.io/rqueue)
* Releases: [https://github.com/sonus21/rqueue/releases](https://github.com/sonus21/rqueue/releases)
* Issue tracker: [https://github.com/sonus21/rqueue/issues](https://github.com/sonus21/rqueue/issues)
* Maven Central:
* [https://repo1.maven.org/maven2/com/github/sonus21/rqueue-spring](https://repo1.maven.org/maven2/com/github/sonus21/rqueue-spring)
* [https://repo1.maven.org/maven2/com/github/sonus21/rqueue-spring-boot-starter](https://repo1.maven.org/maven2/com/github/sonus21/rqueue-spring-boot-starter)
* StackOverflow: [https://stackoverflow.com/tags/rqueue](https://stackoverflow.com/tags/rqueue)
* Discord: https://discord.gg/2hgQZXS2
* Project Progress: [https://github.com/sonus21/rqueue/projects/3](https://github.com/sonus21/rqueue/projects/3)
## License
© [Sonu Kumar](mailto:[email protected]) 2019-Instant.now
The Rqueue is released under version 2.0 of the Apache License.