{"id":13658469,"url":"https://github.com/allegro/node-worker-nodes","last_synced_at":"2025-04-12T20:41:19.899Z","repository":{"id":14686226,"uuid":"76860538","full_name":"allegro/node-worker-nodes","owner":"allegro","description":"A node.js library to run cpu-intensive tasks in a separate processes and not block the event loop.","archived":false,"fork":false,"pushed_at":"2024-09-17T13:38:14.000Z","size":441,"stargazers_count":494,"open_issues_count":12,"forks_count":33,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-04-05T19:08:41.902Z","etag":null,"topics":["cluster","hacktoberfest","nodejs","processes","workers"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/allegro.png","metadata":{"files":{"readme":"Readme.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-12-19T12:31:12.000Z","updated_at":"2025-01-16T01:38:54.000Z","dependencies_parsed_at":"2024-01-15T20:47:48.181Z","dependency_job_id":"d40167e1-9f02-4e14-b56b-55ab5aa34ce5","html_url":"https://github.com/allegro/node-worker-nodes","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allegro%2Fnode-worker-nodes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allegro%2Fnode-worker-nodes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allegro%2Fnode-worker-nodes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allegro%2Fnode-worker-nodes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/allegro","download_url":"https://codeload.github.com/allegro/node-worker-nodes/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248631669,"owners_count":21136554,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster","hacktoberfest","nodejs","processes","workers"],"created_at":"2024-08-02T05:00:59.825Z","updated_at":"2025-04-12T20:41:19.871Z","avatar_url":"https://github.com/allegro.png","language":"JavaScript","readme":"[![view on npm](https://img.shields.io/npm/v/worker-nodes.svg)](https://www.npmjs.org/package/worker-nodes)\n[![Build status](https://github.com/allegro/node-worker-nodes/actions/workflows/ci.yml/badge.svg)](https://github.com/allegro/node-worker-nodes/actions/workflows/ci.yml)\n\n# worker-nodes\n\n  A node.js library to run cpu-intensive tasks in a separate processes and to not to block the event loop.\n\n\n## Installation\n\n```bash\n$ npm install worker-nodes\n```\n\n  Node.js greater than 14.0.0 is *required*\n\n# API Reference\n\n\u003ca name=\"WorkerNodes\"\u003e\u003c/a\u003e\n\n## WorkerNodes\n**Kind**: global class  \n\n* [WorkerNodes](#WorkerNodes)\n    * [new WorkerNodes(path, [options])](#new_WorkerNodes_new)\n    * [.call](#WorkerNodes+call) : \u003ccode\u003eProxy\u003c/code\u003e\n    * [.ready()](#WorkerNodes+ready) ⇒ \u003ccode\u003ePromise\u003c/code\u003e\n    * [.terminate()](#WorkerNodes+terminate) ⇒ \u003ccode\u003ePromise\u003c/code\u003e\n    * [.profiler(duration)](#WorkerNodes+profiler) ⇒ \u003ccode\u003evoid\u003c/code\u003e\n    * [.takeSnapshot()](#WorkerNodes+takeSnapshot) ⇒ \u003ccode\u003evoid\u003c/code\u003e\n    * [.getUsedWorkers()](#WorkerNodes+getUsedWorkers) ⇒ \u003ccode\u003eArray.\u0026lt;Worker\u0026gt;\u003c/code\u003e\n\n\u003ca name=\"new_WorkerNodes_new\"\u003e\u003c/a\u003e\n\n### new WorkerNodes(path, [options])\n\n| Param | Type | Description |\n| --- | --- | --- |\n| path | \u003ccode\u003eString\u003c/code\u003e | An absolute path to the module that will be run in the workers. |\n| [options] | \u003ccode\u003eObject\u003c/code\u003e | See [WorkerNodesOptions](#WorkerNodesOptions) for a detailed description. |\n\n\u003ca name=\"WorkerNodes+call\"\u003e\u003c/a\u003e\n\n### workerNodes.call : \u003ccode\u003eProxy\u003c/code\u003e\nThis exposes the api of a module that the worker nodes are working on. If the module is a function, you\ncan call this directly. If the module exports multiple functions, you can call them as they were properties\nof this proxy.\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodes\u003c/code\u003e](#WorkerNodes)  \n\u003ca name=\"WorkerNodes+ready\"\u003e\u003c/a\u003e\n\n### workerNodes.ready() ⇒ \u003ccode\u003ePromise\u003c/code\u003e\nA method to check if the minimum required number of workers are ready to serve the calls.\n\n**Kind**: instance method of [\u003ccode\u003eWorkerNodes\u003c/code\u003e](#WorkerNodes)  \n**Returns**: \u003ccode\u003ePromise\u003c/code\u003e - resolves with a [WorkerNodes](#WorkerNodes) instance  \n\u003ca name=\"WorkerNodes+terminate\"\u003e\u003c/a\u003e\n\n### workerNodes.terminate() ⇒ \u003ccode\u003ePromise\u003c/code\u003e\nStarts the process of terminating this instance.\n\n**Kind**: instance method of [\u003ccode\u003eWorkerNodes\u003c/code\u003e](#WorkerNodes)  \n**Returns**: \u003ccode\u003ePromise\u003c/code\u003e - - resolved when the instance is terminated.  \n\u003ca name=\"WorkerNodes+profiler\"\u003e\u003c/a\u003e\n\n### workerNodes.profiler(duration) ⇒ \u003ccode\u003evoid\u003c/code\u003e\nRun CPU Profiler and save result on main process directory\n\n**Kind**: instance method of [\u003ccode\u003eWorkerNodes\u003c/code\u003e](#WorkerNodes)  \n\n| Param | Type |\n| --- | --- |\n| duration | \u003ccode\u003enumber\u003c/code\u003e | \n\n\u003ca name=\"WorkerNodes+takeSnapshot\"\u003e\u003c/a\u003e\n\n### workerNodes.takeSnapshot() ⇒ \u003ccode\u003evoid\u003c/code\u003e\nTake Heap Snapshot and save result on main process directory\n\n**Kind**: instance method of [\u003ccode\u003eWorkerNodes\u003c/code\u003e](#WorkerNodes)  \n\u003ca name=\"WorkerNodes+getUsedWorkers\"\u003e\u003c/a\u003e\n\n### workerNodes.getUsedWorkers() ⇒ \u003ccode\u003eArray.\u0026lt;Worker\u0026gt;\u003c/code\u003e\nReturn list with used workers in pool\n\n**Kind**: instance method of [\u003ccode\u003eWorkerNodes\u003c/code\u003e](#WorkerNodes)  \n\n\u003ca name=\"WorkerNodesOptions\"\u003e\u003c/a\u003e\n\n## WorkerNodesOptions\nDescribes a WorkerNodes options.\n\n**Kind**: global class  \n\n* [WorkerNodesOptions](#WorkerNodesOptions)\n    * [.autoStart](#WorkerNodesOptions+autoStart) : \u003ccode\u003eBoolean\u003c/code\u003e\n    * [.lazyStart](#WorkerNodesOptions+lazyStart) : \u003ccode\u003eBoolean\u003c/code\u003e\n    * [.asyncWorkerInitialization](#WorkerNodesOptions+asyncWorkerInitialization) : \u003ccode\u003eBoolean\u003c/code\u003e\n    * [.minWorkers](#WorkerNodesOptions+minWorkers) : \u003ccode\u003eNumber\u003c/code\u003e\n    * [.maxWorkers](#WorkerNodesOptions+maxWorkers) : \u003ccode\u003eNumber\u003c/code\u003e\n    * [.maxTasks](#WorkerNodesOptions+maxTasks) : \u003ccode\u003eNumber\u003c/code\u003e\n    * [.maxTasksPerWorker](#WorkerNodesOptions+maxTasksPerWorker) : \u003ccode\u003eNumber\u003c/code\u003e\n    * [.taskTimeout](#WorkerNodesOptions+taskTimeout) : \u003ccode\u003eNumber\u003c/code\u003e\n    * [.taskMaxRetries](#WorkerNodesOptions+taskMaxRetries) : \u003ccode\u003eNumber\u003c/code\u003e\n    * [.workerEndurance](#WorkerNodesOptions+workerEndurance) : \u003ccode\u003eNumber\u003c/code\u003e\n    * [.workerStopTimeout](#WorkerNodesOptions+workerStopTimeout) : \u003ccode\u003eNumber\u003c/code\u003e\n    * [.resourceLimits](#WorkerNodesOptions+resourceLimits) : \u003ccode\u003eObject\u003c/code\u003e\n    * [.workerType](#WorkerNodesOptions+workerType) : \u003ccode\u003estring\u003c/code\u003e\n\n\u003ca name=\"WorkerNodesOptions+autoStart\"\u003e\u003c/a\u003e\n\n### options.autoStart : \u003ccode\u003eBoolean\u003c/code\u003e\nWhether should initialize the workers before a first call.\n\nIf true, depending on the [lazyStart](#WorkerNodesOptions+lazyStart) option, it will start the\n[min](#WorkerNodesOptions+minWorkers) or [max](#WorkerNodesOptions+maxWorkers) number of workers.\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodesOptions\u003c/code\u003e](#WorkerNodesOptions)  \n**Default**: \u003ccode\u003efalse\u003c/code\u003e  \n\u003ca name=\"WorkerNodesOptions+lazyStart\"\u003e\u003c/a\u003e\n\n### options.lazyStart : \u003ccode\u003eBoolean\u003c/code\u003e\nWhether should start a new worker only if all the others are busy.\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodesOptions\u003c/code\u003e](#WorkerNodesOptions)  \n**Default**: \u003ccode\u003efalse\u003c/code\u003e  \n\u003ca name=\"WorkerNodesOptions+asyncWorkerInitialization\"\u003e\u003c/a\u003e\n\n### options.asyncWorkerInitialization : \u003ccode\u003eBoolean\u003c/code\u003e\nEnables async initialization of worker.\nTo start handling task over worker, need to invoke `sendWorkerMessage('ready')` function when it fully initialized.\nFor examples please refer to [the test cases](https://github.com/allegro/node-worker-nodes/blob/master/e2e/async-initialization.spec.js)\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodesOptions\u003c/code\u003e](#WorkerNodesOptions)  \n**Default**: \u003ccode\u003efalse\u003c/code\u003e  \n\u003ca name=\"WorkerNodesOptions+minWorkers\"\u003e\u003c/a\u003e\n\n### options.minWorkers : \u003ccode\u003eNumber\u003c/code\u003e\nThe minimum number of workers that needs to be running to consider the whole pool as operational.\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodesOptions\u003c/code\u003e](#WorkerNodesOptions)  \n**Default**: \u003ccode\u003e0\u003c/code\u003e  \n\u003ca name=\"WorkerNodesOptions+maxWorkers\"\u003e\u003c/a\u003e\n\n### options.maxWorkers : \u003ccode\u003eNumber\u003c/code\u003e\nThe maximum number of workers that can be running at the same time.\nDefaults to the number of cores the operating system sees.\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodesOptions\u003c/code\u003e](#WorkerNodesOptions)  \n\u003ca name=\"WorkerNodesOptions+maxTasks\"\u003e\u003c/a\u003e\n\n### options.maxTasks : \u003ccode\u003eNumber\u003c/code\u003e\nThe maximum number of calls that can be handled at the same time.\nExceeding this limit causes MaxConcurrentCallsError to be thrown.\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodesOptions\u003c/code\u003e](#WorkerNodesOptions)  \n**Default**: \u003ccode\u003eInfinity\u003c/code\u003e  \n\u003ca name=\"WorkerNodesOptions+maxTasksPerWorker\"\u003e\u003c/a\u003e\n\n### options.maxTasksPerWorker : \u003ccode\u003eNumber\u003c/code\u003e\nThe number of calls that can be given to a single worker at the same time.\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodesOptions\u003c/code\u003e](#WorkerNodesOptions)  \n**Default**: \u003ccode\u003e1\u003c/code\u003e  \n\u003ca name=\"WorkerNodesOptions+taskTimeout\"\u003e\u003c/a\u003e\n\n### options.taskTimeout : \u003ccode\u003eNumber\u003c/code\u003e\nThe number milliseconds after which a call is considered to be lost.\nExceeding this limit causes TimeoutError to be thrown and a worker that performed that task to be killed.\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodesOptions\u003c/code\u003e](#WorkerNodesOptions)  \n**Default**: \u003ccode\u003eInfinity\u003c/code\u003e  \n\u003ca name=\"WorkerNodesOptions+taskMaxRetries\"\u003e\u003c/a\u003e\n\n### options.taskMaxRetries : \u003ccode\u003eNumber\u003c/code\u003e\nThe maximum number of retries that will be performed over a task before reporting it as incorrectly terminated.\nExceeding this limit causes ProcessTerminatedError to be thrown.\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodesOptions\u003c/code\u003e](#WorkerNodesOptions)  \n**Default**: \u003ccode\u003e0\u003c/code\u003e  \n\u003ca name=\"WorkerNodesOptions+workerEndurance\"\u003e\u003c/a\u003e\n\n### options.workerEndurance : \u003ccode\u003eNumber\u003c/code\u003e\nThe maximum number of calls that a single worker can handle during its whole lifespan.\nExceeding this limit causes the termination of the worker.\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodesOptions\u003c/code\u003e](#WorkerNodesOptions)  \n**Default**: \u003ccode\u003eInfinity\u003c/code\u003e  \n\u003ca name=\"WorkerNodesOptions+workerStopTimeout\"\u003e\u003c/a\u003e\n\n### options.workerStopTimeout : \u003ccode\u003eNumber\u003c/code\u003e\nThe timeout value (in milliseconds) for the worker to stop before sending SIGKILL.\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodesOptions\u003c/code\u003e](#WorkerNodesOptions)  \n**Default**: \u003ccode\u003e100\u003c/code\u003e  \n\u003ca name=\"WorkerNodesOptions+resourceLimits\"\u003e\u003c/a\u003e\n\n### options.resourceLimits : \u003ccode\u003eObject\u003c/code\u003e\nProvides the set of JS engine resource constraints inside this Worker thread. (Usable when using `workerType: thread` only)\n\n**Kind**: instance property of [\u003ccode\u003eWorkerNodesOptions\u003c/code\u003e](#WorkerNodesOptions)  \n**Properties**\n\n| Name | Type | Description |\n| --- | --- | --- |\n| maxYoungGenerationSizeMb | \u003ccode\u003eNumber\u003c/code\u003e | The maximum size of a heap space for recently created objects |\n| maxOldGenerationSizeMb | \u003ccode\u003eNumber\u003c/code\u003e | The maximum size of the main heap in MB |\n| codeRangeSizeMb | \u003ccode\u003eNumber\u003c/code\u003e | The size of a pre-allocated memory range used for generated code |\n| stackSizeMb | \u003ccode\u003eNumber\u003c/code\u003e | The default maximum stack size for the thread. Small values may lead to unusable Worker instances |\n\n### options.workerType : \u003ccode\u003estring\u003c/code\u003e\nCan be either `process` or `thread` (default), that controls the underlying implementation used, either `child_process` or `worker_threads`.\nMost usecases are perfectly fine with `thread` implementation, some work loads though, might need to use `process`, for example, if you are using\n`process.chdir()` call which is [not supported](https://github.com/nodejs/node/issues/41673) in `worker_threads`.\n\n## Example\n\nGiven `/home/joe.doe/workspace/my-module.js`:\n```javascript\nmodule.exports = function myTask() {\n    return 'hello from separate process!';\n};\n```\n\nyou can run it through the worker nodes as follows:\n```javascript\nconst WorkerNodes = require('worker-nodes');\nconst myModuleWorkerNodes = new WorkerNodes('/home/joe.doe/workspace/my-module');\n\nmyModuleWorkerNodes.call().then(msg =\u003e console.log(msg));  // -\u003e 'hello from separate process!'\n```\n\nFor more advanced examples please refer to [the test cases](https://github.com/allegro/node-worker-nodes/tree/main/e2e).\n\n\n## Running tests\n\nCheck out the library code and then:\n\n```bash\n$ npm install\n$ npm test\n```\n\n## Benchmarks\n\nTo run tests, type:\n\n```bash\n$ npm install\n$ npm run benchmark\n```\n\nIt will run a performance test against the selected libraries:\n- data in: an object that consists of a single field that is a 0.5MB random string\n- data out: received object stringified and concatenated with another 1MB string\n\nExample results:\n```bash\nresults for 100 executions\n\nname                time: total [ms]  time usr [ms]  time sys [ms]  worker usr [ms]  worker sys [ms]  mem rss [MB]  worker rss [MB]  errors\n------------------  ----------------  -------------  -------------  ---------------  ---------------  ------------  ---------------  ------\nno-workers                       148            203             37                0                0            98                0       0\nworker-nodes@2.0.0               362            390            143              389              143           213              210       0\nworkerpool@6.0.0                 367            495            185              492              182           236              245       0\nworker-nodes@1.6.1              1095            520            207              592              243           216               86       0\nworker-farm@1.7.0               1886            749            276              947              299           221               70       0\nprocess-pool@0.3.5              2002            847            285              986              309           219               74       0\nworker-pool@3.0.2              13775           7129           5236             1891              952           363               63       0\n\n  os : Darwin / 19.5.0 / x64\n cpu : Intel(R) Core(TM) i7-7660U CPU @ 2.50GHz × 4\nnode : 14.3.0 / v8: 8.1.307.31-node.33\n```\n\n## See also\n\nsources of inspiration:\n* [Worker Farm](https://github.com/rvagg/node-worker-farm)\n* [workerpool](https://github.com/josdejong/workerpool)\n* [process-pool](https://github.com/ohjames/process-pool)\n* [node-worker-pool](https://github.com/jeffmo/node-worker-pool)\n\n## License\n\nCopyright Allegro Sp. z o.o.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallegro%2Fnode-worker-nodes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fallegro%2Fnode-worker-nodes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallegro%2Fnode-worker-nodes/lists"}