Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/taskforcesh/bullmq
BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis
https://github.com/taskforcesh/bullmq
background-jobs nodejs python queue redis typescript
Last synced: 11 days ago
JSON representation
BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis
- Host: GitHub
- URL: https://github.com/taskforcesh/bullmq
- Owner: taskforcesh
- License: mit
- Created: 2018-12-19T21:46:28.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2024-10-12T05:21:19.000Z (27 days ago)
- Last Synced: 2024-10-12T08:01:54.022Z (27 days ago)
- Topics: background-jobs, nodejs, python, queue, redis, typescript
- Language: TypeScript
- Homepage: https://bullmq.io
- Size: 73 MB
- Stars: 6,038
- Watchers: 33
- Forks: 391
- Open Issues: 326
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: contributing.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
- awesome - taskforcesh/bullmq - BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis (TypeScript)
- awesome-nodejs - bullmq - BullMQ - Premium Message Queue for NodeJS based on Redis.  (Repository / Job Queues)
- awesome-nodejs-cn - bullmq - **star:5548** 持久作业和消息队列 ![star > 2000][Awesome] (包 / 工作队列)
- awesome-nodejs - bullmq - Persistent job and message queue. (Packages / Job queues)
- awesome-star-libs - taskforcesh / bullmq - 基于 Redis 的 NodeJS 和 Python 消息队列和批处理功能 (目录)
README
The fastest, most reliable, Redis-based distributed queue for Node.
Carefully written for rock solid stability and atomicity.
Read the documentation
Follow @manast for *important* Bull/BullMQ/BullMQ-Pro news and updates!
# 🛠 Tutorials
You can find tutorials and news in this blog: https://blog.taskforce.sh/
# News 🚀
## 🌐 Language agnostic BullMQ
Do you need to work with BullMQ on platforms other than Node.js? If so, check out the [BullMQ Proxy](https://github.com/taskforcesh/bullmq-proxy)
## 🌟 Rediscover Scale Conference 2024
Discover the latest in in-memory and real-time data technologies at **Rediscover Scale 2024**. Ideal for engineers, architects, and technical leaders looking to push technological boundaries. Connect with experts and advance your skills at The Foundry SF, San Francisco.
[Learn more and register here!](https://www.rediscoverscale.com/)
# Official FrontEnd
[
](https://taskforce.sh)
Supercharge your queues with a professional front end:
- Get a complete overview of all your queues.
- Inspect jobs, search, retry, or promote delayed jobs.
- Metrics and statistics.
- and many more features.
Sign up at [Taskforce.sh](https://taskforce.sh)
# 🚀 Sponsors 🚀
Dragonfly is a new Redis™ drop-in replacement that is fully compatible with BullMQ and brings some important advantages over Redis™ such as massive
better performance by utilizing all CPU cores available and faster and more memory efficient data structures. Read more here on how to use it with BullMQ.
# Used by
Some notable organizations using BullMQ:
# The gist
Install:
```
$ yarn add bullmq
```
Add jobs to the queue:
```ts
import { Queue } from 'bullmq';
const queue = new Queue('Paint');
queue.add('cars', { color: 'blue' });
```
Process the jobs in your workers:
```ts
import { Worker } from 'bullmq';
const worker = new Worker('Paint', async job => {
if (job.name === 'cars') {
await paintCar(job.data.color);
}
});
```
Listen to jobs for completion:
```ts
import { QueueEvents } from 'bullmq';
const queueEvents = new QueueEvents('Paint');
queueEvents.on('completed', ({ jobId }) => {
console.log('done painting');
});
queueEvents.on(
'failed',
({ jobId, failedReason }: { jobId: string; failedReason: string }) => {
console.error('error painting', failedReason);
},
);
```
This is just scratching the surface, check all the features and more in the official documentation
# Feature Comparison
Since there are a few job queue solutions, here is a table comparing them:
| Feature | [BullMQ-Pro](https://bullmq.io/#bullmq-pro) | [BullMQ](https://bullmq.io) | Bull | Kue | Bee | Agenda |
| :------------------------ | :-------------: | :-------------: | :-------------: | :---: | -------- | ------ |
| Backend | redis | redis | redis | redis | redis | mongo |
| Observables | ✓ | | | | | |
| Group Rate Limit | ✓ | | | | | |
| Group Support | ✓ | | | | | |
| Batches Support | ✓ | | | | | |
| Parent/Child Dependencies | ✓ | ✓ | | | | |
| Debouncing | ✓ | ✓ | ✓ | | | |
| Priorities | ✓ | ✓ | ✓ | ✓ | | ✓ |
| Concurrency | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Delayed jobs | ✓ | ✓ | ✓ | ✓ | | ✓ |
| Global events | ✓ | ✓ | ✓ | ✓ | | |
| Rate Limiter | ✓ | ✓ | ✓ | | | |
| Pause/Resume | ✓ | ✓ | ✓ | ✓ | | |
| Sandboxed worker | ✓ | ✓ | ✓ | | | |
| Repeatable jobs | ✓ | ✓ | ✓ | | | ✓ |
| Atomic ops | ✓ | ✓ | ✓ | | ✓ | |
| Persistence | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| UI | ✓ | ✓ | ✓ | ✓ | | ✓ |
| Optimized for | Jobs / Messages | Jobs / Messages | Jobs / Messages | Jobs | Messages | Jobs |
## Contributing
Fork the repo, make some changes, submit a pull-request! Here is the [contributing](contributing.md) doc that has more details.
# Thanks
Thanks for all the contributors that made this library possible,
also a special mention to Leon van Kammen that kindly donated
his npm bullmq repo.