{"id":20315794,"url":"https://github.com/flipeador/node-workerpool","last_synced_at":"2026-05-13T02:05:19.188Z","repository":{"id":157621919,"uuid":"547268661","full_name":"flipeador/node-workerpool","owner":"flipeador","description":"Create a pool of workers and execute a function or file in worker threads and child processes.","archived":false,"fork":false,"pushed_at":"2024-01-06T21:09:12.000Z","size":13,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-04T09:12:57.624Z","etag":null,"topics":["async-await","child-process","javascript","nodejs","worker-pool","worker-threads"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flipeador.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-10-07T12:05:01.000Z","updated_at":"2025-02-14T06:10:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"75b52b71-8cb5-4144-9673-7214cd024bee","html_url":"https://github.com/flipeador/node-workerpool","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/flipeador/node-workerpool","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flipeador%2Fnode-workerpool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flipeador%2Fnode-workerpool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flipeador%2Fnode-workerpool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flipeador%2Fnode-workerpool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flipeador","download_url":"https://codeload.github.com/flipeador/node-workerpool/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flipeador%2Fnode-workerpool/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32964445,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-12T23:30:32.555Z","status":"online","status_checked_at":"2026-05-13T02:00:07.132Z","response_time":115,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async-await","child-process","javascript","nodejs","worker-pool","worker-threads"],"created_at":"2024-11-14T18:21:48.955Z","updated_at":"2026-05-13T02:05:19.164Z","avatar_url":"https://github.com/flipeador.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Node.js WorkerPool\n\nCreate a pool of workers and execute a function or file in [worker threads](https://nodejs.org/api/worker_threads.html) and [child processes](https://nodejs.org/api/child_process.html).\n\n### Features\n\n- Workers are created only when needed and are terminated after a certain period of inactivity.\n- If a successfully loaded worker is terminated by any of the following reasons, a new one will be created as necessary.\n  - The worker is explicitly terminated by a call to [ChildProcess#kill](https://nodejs.org/api/child_process.html#subprocesskillsignal) or [Worker#terminate](https://nodejs.org/api/worker_threads.html#workerterminate).\n  - The worker has terminated due to an unhandled exception.\n  - A child process has been terminated by external means.\n- Supports bidirectional communication with the worker.\n- Post messages and send messages waiting for a response.\n- Execute functions on the worker and force termination if the timeout has expired.\n- Lightweight with full async-await support and timeouts.\n\n### Transfer complex objects between workers\n\nThere are certain limitations concerning the data that can be transferred between threads and processes. Complex objects are transferred by using the [HTML structured clone algorithm](https://developer.mozilla.org/docs/Web/API/Web_Workers_API/Structured_clone_algorithm).\n\nThrown exceptions are automatically serialized as follows:\n\n```js\nclass MyError extends Error { prop = 255; }\nconst error = new MyError('exception');\nconst serialized = {error, props: Object.assign({}, error)};\nconst deserialized = Object.assign(serialized.error, serialized.props);\nconsole.log(deserialized);\n```\n\n## Installation\n\n```\nnpm i flipeador/node-workerpool#semver:^1.0.0\n```\n\n## pseudo-Documentation\n\nStart by instantiating a `WorkerPool` object.\n\n- `script` — The module to run in the worker thread or child process. Worker threads also accept js code or a function; If js code is specified, it is placed inside a function called `main`.\n- `options` — Options.\n  - `workerType` — Worker type: `'thread'` or `'process'`. Defaults to `'thread'`.\n  - `workerOptions` — Options passed to [child_process.fork](https://nodejs.org/api/child_process.html#child_processforkmodulepath-args-options) or [worker_threads.Worker](https://nodejs.org/api/worker_threads.html#class-worker).\n  - `maxWorkers` — Maximum number of concurrent workers. Defaults to the maximum number of [logical CPU cores](https://nodejs.org/api/os.html#oscpus) minus 1.\n  - `maxQueueSize` — Maximum number of tasks allowed to be queued. Defaults to `Infinity`.\n  - `exitTimeout` — [MS] Determines how long to wait before a worker is terminated due to inactivity. Defaults to 5 minutes.\n  - `heapSizeLimit` — [MB] Maximum memory usage for child processes ([--max-old-space-size](https://nodejs.org/api/cli.html#--max-old-space-sizesize-in-megabytes)).\n\n```js\nconst workerpool = new WorkerPool(script, options);\n```\n\nCreate a task that allows bidirectional communication with a worker.\n\nThe `WorkerPool#task` method returns a promise that resolves to a `Task` object once an available worker has been associated with the task.\n\n```js\nconst task = await workerpool.task();\n```\n\nWhen the task is no longer needed, it is important to call the `Task#done` method to release the associated worker, so that other tasks can reuse it.\n\nThis method can be called at any time, the worker will not be released until all the requests have been handled, but once called, the `Task` object can no longer be used for further communication.\n\nThe `Task#done` method returns a promise that resolves once the associated worker is terminated or becomes available for another task.\nIf the promise is taking too long to resolve, worker termination can be forced by calling `Task#terminate`, but this will cancel any pending response.\n\n```js\n// returns undefined if the worker still exists\nconst exitCode = await task.done();\n```\n\n---\n\nIf a module has been passed to `WorkerPool`, the worker module must instantiate a `Worker` object.\n\n- The `WorkerPool` and `Worker` objects extends [EventEmitter](https://github.com/flipeador/node-eventemitter), so listener functions can be registered.\n- The `Worker#init` method initializes the module to enable communication with `WorkerPool`.\n\n```js\nnew Worker()\n.on('message', async (message) =\u003e {\n    // do something with the message\n    return 'response';\n})\n.init({\n    // functions available for execution\n    main(...args) {\n        return 'result';\n    }\n});\n```\n\n---\n\nSee [examples](#examples) for more information.\n\n## Examples\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch4\u003eDemonstrates the behavior of the workerpool\u003c/h4\u003e\u003c/summary\u003e\n\n```js\nimport process from 'node:process';\nimport readline from 'node:readline';\nimport url from 'node:url';\n\nimport { WorkerPool, Worker } from '@flipeador/node-workerpool';\n\nconst __filename = url.fileURLToPath(import.meta.url);\n\nfunction pause(message) {\n    const rl = readline.createInterface(process.stdin, process.stdout);\n    return new Promise(resolve =\u003e {\n        rl.question(\n            `${message}\\nPress any key to continue...`,\n            answer =\u003e rl.close(resolve(answer))\n        );\n    });\n}\n\nif (Worker.isMainProcess())\n{\n    const workerpool = new WorkerPool(__filename, {\n        workerType: 'process',\n        maxWorkers: 3\n    });\n\n    await pause(`Open the task manager and locate process id ${process.pid}.`);\n\n    while (true) {\n        const ptask = workerpool.task();\n        if (!ptask.worker) console.log(\n            'Waiting for a worker to become available.'\n          + '\\nTry to terminate a child process from the task manager.'\n        );\n        const task = await ptask;\n        await pause(`A new child process has been created (${task.id}).`);\n    }\n}\nelse\n{\n    new Worker().init();\n}\n```\n\n```\nOpen the task manager and locate process id 12384.\nPress any key to continue...\nA new child process has been created (13540).\nPress any key to continue...\nA new child process has been created (4444).\nPress any key to continue...\nA new child process has been created (6972).\nPress any key to continue...\nWaiting for a worker to become available.\nTry to terminate a child process from the task manager.\nA new child process has been created (9700).\nPress any key to continue...\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch4\u003eExecute a function in a worker thread\u003c/h4\u003e\u003c/summary\u003e\n\n\u003e [!WARNING]\n\u003e This is currently not supported for child processes.\n\n```js\nimport { WorkerPool } from '@flipeador/node-workerpool';\n\nasync function worker(param)\n{\n    // Listener functions can be async if needed.\n    this.on('message', m =\u003e `${m} received!`); // (1)\n    this.once('done', () =\u003e console.log('Done!')); // (2)\n\n    const result = await this.send('Hello'); // (3)\n    return result[0] + param;\n}\n\nconst workerpool = new WorkerPool(worker, {\n    exitTimeout: 1000\n});\n\nconst task = await workerpool.task();\n\ntask.on('message', async (message) =\u003e {\n    return `${message} World`; // (3)\n});\n\ntry {\n    // 'execute' returns the result, or throws an exception in case of error.\n    console.log(await task.execute('worker', ['!'])); // (2) 'Hello World!'\n    // 'send' returns a list of results, including exceptions.\n    console.log(await task.send('message')); // (1) [ 'message received!' ]\n} finally {\n    task.done();\n}\n```\n\n```\nHello World!\n[ 'message received!' ]\nDone!\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch4\u003eExecute JavaScript code in a worker thread\u003c/h4\u003e\u003c/summary\u003e\n\n\u003e [!WARNING]\n\u003e This is currently not supported for child processes.\n\n```js\nimport { WorkerPool } from '@flipeador/node-workerpool';\n\nconst script = 'return 3 * args[0];';\n\nconst workerpool = new WorkerPool(script, {\n    exitTimeout: 1000\n});\n\nconst task = await workerpool.task();\n\ntask.execute('main', [5])\n    .then(console.log, console.error);\ntask.done();\n```\n\n```\n15\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch4\u003eExecute a module in a child process\u003c/h4\u003e\u003c/summary\u003e\n\n```js\nimport url from 'node:url';\n\nimport { WorkerPool, Worker } from '@flipeador/node-workerpool';\n\nconst __filename = url.fileURLToPath(import.meta.url);\n\nif (Worker.isMainProcess())\n{\n    const workerpool = new WorkerPool(__filename, {\n        workerType: 'process',\n        exitTimeout: 1000\n    });\n\n    const task = await workerpool.task();\n\n    task.post('Child Process'); // (1)\n    task.execute('main', ['Main', 'Process'])\n        .then(console.log, console.error); // (2)\n    task.done();\n}\nelse\n{\n    new Worker()\n        .on('message', message =\u003e {\n            console.log(`${message}!`); // (1)\n        })\n        .init({\n            main(main, process) {\n                return `${main} ${process}!`; // (2)\n            }\n        });\n}\n```\n\n```\nChild Process!\nMain Process!\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch4\u003eExecute a worker thread from a child process\u003c/h4\u003e\u003c/summary\u003e\n\n```js\nimport url from 'node:url';\n\nimport { WorkerPool, Worker } from '@flipeador/node-workerpool';\n\nconst __filename = url.fileURLToPath(import.meta.url);\n\nif (Worker.isMainProcess())\n{\n    console.log('Main Process');\n\n    const workerpool = new WorkerPool(__filename, {\n        workerType: 'process',\n        exitTimeout: 1000\n    });\n\n    const task = await workerpool.task();\n\n    task.execute('main')\n        .then(console.log, console.error);\n    task.done();\n}\nelse\n{\n    console.log('↳Child Process');\n\n    new Worker().init({\n        async main() {\n            function worker()\n            {\n                console.log(' ↳Worker Thread');\n                return 'Hello World!';\n            }\n\n            const workerpool = new WorkerPool(worker);\n            const task = await workerpool.task();\n\n            try {\n                return await task.execute('worker');\n            } finally {\n                task.done();\n            }\n        }\n    });\n}\n```\n\n```\nMain Process\n↳Child Process\n ↳Worker Thread\nHello World!\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch4\u003eLimit the maximum memory usage of child processes\u003c/h4\u003e\u003c/summary\u003e\n\n```js\nimport v8 from 'node:v8';\nimport url from 'node:url';\n\nimport { WorkerPool, Worker} from '@flipeador/node-workerpool';\n\nconst __filename = url.fileURLToPath(import.meta.url);\n\nif (Worker.isMainProcess())\n{\n    const workerpool = new WorkerPool(__filename, {\n        workerType: 'process',\n        heapSizeLimit: 500, // ~500 MB\n        exitTimeout: 1000\n    });\n    const task = await workerpool.task();\n    task.done();\n}\nelse\n{\n    const stats = v8.getHeapStatistics();\n    console.log('Used heap size:', Math.round(stats.used_heap_size/1024**2), 'MB');\n    console.log('Heap size limit:', Math.round(stats.heap_size_limit/1024**2), 'MB');\n    new Worker().init();\n}\n```\n\n```\nUsed heap size: 6 MB\nHeap size limit: 548 MB\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch4\u003eSpawn an executable file as a new child process\u003c/h4\u003e\u003c/summary\u003e\n\nThis is a convenient promisified version of [child_process.execFile](https://nodejs.org/api/child_process.html#child_processexecfilefile-args-options-callback).\n\n```js\nimport { execPath } from 'node:process';\nimport { Process } from '@flipeador/node-workerpool';\n\nconsole.log(execPath);\nProcess.execute(execPath, ['--version'], {\n    //data: 'stdin', timeout: 1000\n}).then(console.log);\n```\n\n\u003c/details\u003e\n\n## License\n\nThis project is licensed under the **Apache License 2.0**. See the [license file](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflipeador%2Fnode-workerpool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflipeador%2Fnode-workerpool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflipeador%2Fnode-workerpool/lists"}