https://github.com/thinkmill/node-worker
🎡 A simple, promise-based, worker framework.
https://github.com/thinkmill/node-worker
nodejs worker
Last synced: 22 days ago
JSON representation
🎡 A simple, promise-based, worker framework.
- Host: GitHub
- URL: https://github.com/thinkmill/node-worker
- Owner: Thinkmill
- License: bsd-2-clause
- Created: 2018-02-19T05:40:09.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2021-05-09T15:10:21.000Z (almost 5 years ago)
- Last Synced: 2025-04-12T00:17:10.148Z (10 months ago)
- Topics: nodejs, worker
- Language: JavaScript
- Homepage:
- Size: 40 KB
- Stars: 3
- Watchers: 25
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Worker Framework
================================================================================
A simple, promise-based, worker framework.
Install
--------------------------------------------------------------------------------
```sh
yarn add @thinkmill/node-worker
```
Constructor
--------------------------------------------------------------------------------
| Argument | Type | Description |
|----------|------|-------------|
| `label` | `String` | A label that describes the worker (used in logs, errors, etc) |
| `payload` | `Function` | A promise-returning function that is executed on schedule (received the run ordinal) |
| `options` | `Object` | Options controlling how the worker should behave |
### `payload`
The `payload` function represents the main body of the worker, where the real work is done.
It should return a promise that resolves or rejects withing the `timeoutMs` provided to it.
The `payload` is invoked according to the following schedule:
* 1000 ms after the `worker.start()` function is called
* If the promise **rejects** or returns a **truthy** value, the worker will sleep for it's configured `sleepMs` period before invoking the `payload`
* If the promise returns a **falsey** value the worker will sleep for 1000 ms before invoking the `payload`
When invoked, the `payload` will be provided with a single argument; an object containing the following:
| Property | Type | Description |
|----------|------|-------------|
| `label` | `String` | The worker label that was provided on construction |
| `ordinal` | `Number` | An integer indicating how many times the payload has been executed |
| `timeoutMs` | `Number` | The number of milliseconds the worker will wait for this invocation to return |
### `options`
The `options` can contain:
| Property | Type | Description |
|----------|------|-------------|
| `sleepMs` | `Number` | How long do we pause between runs? (in milliseconds) |
| `timeoutMs` | `Number` | How long do we wait (in milliseconds) for the run promise to resolve/reject? See important notes below! |
Note that the `timeoutMs` provided forces the end of a cycle (and allows the next run to be scheduled) but does not (and cannot?) terminate the still-running promise.
If the promise returned by the `payload` function has errored internally (without resolving or rejecting) then that's OK; the schedule timeout will prevent the worker from stalling forever.
_But_ if the promise returned is still doing work, there's the possibility we'll end up with **multiple instances** of the payload executing in tandem.
This is almost certainly a Bad Thing, but that's up to you.
The take away:
* Make sure your worker promises always resolve
* Probably set a fairly long `timeoutMs` value
### Instance methods
| Method | Description |
| ------ | ----------- |
| `start()` | Start the worker after a short delay. |
| `stop()` | Stop the worker. The currently running job will be allowed to compete but the worker will not restart afterwords. |
| `scheduleRunInMs(delayMs, onceOff`) | Schedule a run in `delayMs` milliseconds. If `onceOff` is `true` this will trigger an extra run rather than rescheduling the next run. If the worker has been stopped this will have no effect. |
Usage
--------------------------------------------------------------------------------
### Examples
A simple example:
```js
const Worker = require('@thinkmill/node-worker');
const myWorker = new Worker(
'test-worker',
({ label, ordinal, timeoutMs }) => {
return new Promise((resolve, reject) => {
const takeMs = 4000 + (Math.random() * 1500);
console.log(`Run #${ordinal} ..
will take ${takeMs} ms`);
setTimeout(() => {
console.log(`Run #${ordinal} ..
resolving`);
resolve(true);
}, takeMs);
});
}, {
sleepMs: 5000,
timeoutMs: 5 * 1000,
}
);
myWorker.start();
```
A more realistic/interesting example, processing items in a queue:
```js
const Worker = require('@thinkmill/node-worker');
const debug = require('debug')('workers:dequeue-things');
const Model = require('../models/queuedThings');
// Manage the dequeuing of things
const payload = async ({ label, ordinal, timeoutMs }) => {
const runForMs = timeoutMs - 1000;
const runUntil = new Date(Date.now() + runForMs);
debug(`Running for ${runForMs} ms (until ${runUntil.toISOString()})`);
let processedCount = 0;
let queueEmptied = false;
let nextThing;
do {
await knex.transaction(async (trx) => {
// Get the next thing from the queue
nextThing = await Model.query(trx).findOne('isReady', true).whereNull('processedAt').orderBy('queuedAt');
// The queue is empty; exit early
if (!nextThing) {
queueEmptied = true;
return;
}
// Do whatever it is that things do
// ..
// Record that we've processed this thing
await Model.query(trx).update({ processedAt: new Date() }).where({ id: nextThing.id });
// Inc. our count
processedCount++;
});
}
while (new Date() < runUntil);
// Output some debug info
const summaryMsg = `DONE: ${processedCount} things processed, leaving the queue ${queueEmptied ? 'EMPTY' : 'NOT EMPTY'}`;
debug(summaryMsg);
// Resolve with a boolean indicating whether the payload should be re-invoked soon or after the normal sleep
return queueEmptied;
};
// Create the worker instance and start it
const worker = new Worker('dequeue-things', payload, { sleepMs: 60 * 1000 });
worker.start();
```
You can also request a once-off worker run from some other action. For example, if you have a worker sending emails which processes a queue every 10 minutes, you might want to trigger that worker after a successful account creation.
```js
class AccountCreator {
onCreateSuccess () {
// Tell the email worker to run so the user receives email promptly.
const onceOff = true;
require('../email-worker').scheduleRunInMs(200, onceOff);
}
}
```
### Debug
We use the [`debug`](https://www.npmjs.com/package/debug) package internally.
Entries scoped to `workers:${label}` (where `label` is that supplied on construction).
It's probably helpful to follow this pattern in your own `payload` functions.
Output can be enabled by supplying a scope (or list of scopes) to output in the `DEBUG` env var.
This can include wildcards. Eg
```sh
# All worker debug
DEBUG=workers:* yarn start
# Debug for a specific worker
DEBUG=workers:send-notifications yarn start
# Debug for several specific workers
DEBUG=workers:send-notifications,workers:send-emails yarn start
```
License
--------------------------------------------------------------------------------
BSD Licensed.
Copyright (c) Thinkmill 2018.