{"id":17497508,"url":"https://github.com/miguelcastillo/workit","last_synced_at":"2026-01-24T00:16:50.077Z","repository":{"id":57399303,"uuid":"95365556","full_name":"MiguelCastillo/workit","owner":"MiguelCastillo","description":"JavaScript utility for managing child processes in node","archived":false,"fork":false,"pushed_at":"2017-12-21T16:07:36.000Z","size":47,"stargazers_count":3,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-24T14:14:00.059Z","etag":null,"topics":["thread","thread-pool","worker","worker-pool","worker-queue"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MiguelCastillo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-06-25T14:40:45.000Z","updated_at":"2018-02-14T03:30:14.000Z","dependencies_parsed_at":"2022-09-12T05:40:55.658Z","dependency_job_id":null,"html_url":"https://github.com/MiguelCastillo/workit","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MiguelCastillo%2Fworkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MiguelCastillo%2Fworkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MiguelCastillo%2Fworkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MiguelCastillo%2Fworkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MiguelCastillo","download_url":"https://codeload.github.com/MiguelCastillo/workit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248252696,"owners_count":21072701,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["thread","thread-pool","worker","worker-pool","worker-queue"],"created_at":"2024-10-19T15:52:14.859Z","updated_at":"2026-01-24T00:16:50.030Z","avatar_url":"https://github.com/MiguelCastillo.png","language":"JavaScript","readme":"# workit\nWorkit is a utility for parallel processing. The approach is that of a thread pool in which a queue of jobs exists and workers pick new jobs as they finish their work. But instead of using threads, we use child processes.\n\nParallel processing is traditionally difficult code to write, and that's exactly the problem workit aims to solve for you. Workit accomplishes this by providing you with abstractions for easily defining a Worker API you interact with, managing the lifecycle of the worker processes, and managing the distribution of the workload. All you have to do is define a Worker API that you interact with via a Worker Pool.\n\nSo how does this thing work?  Well - let's think about the setup starting with the worker. A worker is ultimately the child process that processes your data, and you have a couple of ways to define one.\n\n- A module that exports a function which we refer to as a worker function.\n- A module that exports an API by defining a class that extends `Worker` with methods that process your data. We refer to this as a Worker API.\n\nThen you have a worker pool, which is the component you interact with to talk to the workers. A worker pool exposes several methods, but the two important ones for interacting with worker processes are `send` and `invoke`. You use `send` when interacting with a worker function, and you use `invoke` when you interact with a Worker API.\n\n# Examples\n\n## basic setup\n\nThe most basic setup is when the pool and the worker are both defined in the same file. This is perhaps an overly simplistic setup, but it is available if you need it.\n\n``` javascript\nimport Workit from \"workit\";\n\nif (process.isWorker) {\n  console.log(process.pid, process.isWorker, \"hello world\");\n}\nelse {\n  var workerPool = new Workit.Pool(__filename);\n  workerPool.stop();\n}\n```\n\n## Loading a worker module that exports a worker function\n\nindex.js\n``` javascript\nimport path from \"path\";\nimport Workit from \"workit\";\n\nvar workerPool = new Workit.Pool(path.join(__dirname, \"./worker.js\"));\n\nworkerPool.send(\"hello world\").then(() =\u003e {\n  workerPool.stop();\n});\n```\n\nworker.js\n``` javascript\nexport default function(data, done) {\n  console.log(process.pid, process.isWorker, data);\n  done();\n};\n```\n\nrun it\n```\n$ node index.js\n```\n\n\n## Loading a worker module that exports a worker api\n\nindex.js\n``` javascript\nimport path from \"path\";\nimport Workit from \"workit\";\n\nvar workerPool = new Workit.Pool(path.join(__dirname, \"./worker.js\"));\n\nworkerPool.invoke(\"say\", \"hello world\").then((data) =\u003e {\n  console.log(\"received\", data);\n  workerPool.stop();\n});\n```\n\nworker.js\n``` javascript\nimport Workit from \"workit\";\n\nclass Api extends Workit.Worker {\n  say(data) {\n    console.log(process.pid, process.isWorker, data);\n    return Promise.resolve(\"done\");\n  }\n}\n\nexport default Api;\n```\n\nrun it\n```\n$ node index.js\n```\n\n## Worker process talking to the worker pool\n\nindex.js\n``` javascript\nimport path from \"path\";\nimport Workit from \"workit\";\n\nclass Pool extends Workit.Pool {\n  workerSaid(data) {\n    console.log(process.pid, process.isWorker, data);\n  }\n}\n\nvar workerPool = new Pool(path.join(__dirname, \"./worker.js\"));\n\nworkerPool.invoke(\"say\", \"hello world\").then(() =\u003e {\n  workerPool.stop();\n});\n```\n\nworker.js\n``` javascript\nimport Workit from \"workit\";\n\nclass Api extends Workit.Worker {\n  say(data, done) {\n    console.log(process.pid, process.isWorker, data);\n    this.invoke(\"workerSaid\", \"hello back\");\n    done();\n  }\n}\n\nexport default Api;\n```\n\nrun it\n```\n$ node index.js\n```\n\n\n# API\n\n## Workit\n\nworkit provides you with several different constructs to make parallel processing simple.\n\n- Pool\n- Worker\n- isWorker\n\n## Pool(filename: string, options: {}, child-args: [])\n\nClass that manages the queue of jobs and worker processes. You can extend Pool in order to define methods that worker processes can invoke, which allows you to build a two way communication between parent and child processes. You can take a look at the example below for `Worker process talking to Process pool`. The constructor takes several arguments:\n\n1. The name of the file to spin up as child process.\n2. Settings for the worker pool, which get passed down to the child process.\n3. Child process arguments\n\n``` javascript\nnew Pool(path.join(__dirname, \"./worker.js\"), { size: 2 }, [\"--color\"]);\n```\n\nFor a list of available options for the child process, please take a look at the [nodes child_process.fork](https://nodejs.org/api/child_process.html#child_process_child_process_fork_modulepath_args_options) documentation.\n\nOn top of those options, you can pass in the initial size of the pool by specifying `size` as in the example above.\n\n\n### send(data, worker)\n\nMethod to send data to a Worker function.\n\n- data - payload to be sent to the worker process. It can be anything.\n- worker - optional worker instance that should process the data. Otherwise, the scheduling algorithm will pick the worker for you, which is the default behavior.\n\nThe call returns a promise that resolves when the worker process is done processing the data.\n\nThe example below sends a string to be processed by a worker function.\n``` javascript\nworkerPool.send(\"Hello world\").then(() =\u003e console.log(\"done\"));\n```\n\n### invoke(fn, data, worker)\n\nMethod that invokes a method defined in a Worker API.\n\n- fn - method to be called in the Worker API.\n- data - payload to be sent to the Worker API method (fn). It can be anything.\n- worker - optional worker instance that should process the data. Otherwise, the scheduling algorithm will pick the worker for you, which is the default behavior.\n\nThe call returns a promise that resolves when the worker process is done processing the data.\n\nThe example below sends a string to be process by the method `say` in a worker api.\n``` javascript\nworkerPool.invoke(\"say\", \"Hello world\").then(() =\u003e console.log(\"done\"));\n```\n\n### stop()\n\nMethod that stops all workers from processing further jobs. All workers will finish processing whatever they are doing before they exit.\n\n``` javascript\nworkerPool.stop();\n```\n\n### rejectQueue(error)\n\nMethod the rejects all queued jobs that are pending processing. The jobs are rejected with the given error.\n\n``` javascript\nworkerPool.rejectQueue();\n```\n\n### size(value)\n\nMethod that sets the size of the worker pool. The value is absolute, meaning that if you specify 4 then the pool will have a size of 4 worker processes. If the value is smaller than the current, then the pool is reduced to that size gracefully allowing the worker that are going to be removed finish their work. If the value is bigger than the current, then the pool size is increased and jobs are assigned to each worker as needed.\n\n- value - the new size of the worker pool.\n\n``` javascript\nworkerPool.size(2);\n```\n\n### workers[]\n\nArray of all the workers. The workers in this array are essentially an adapter for interacting with the underlying worker process. You have several methods available to you that allow you to interact directly with specific worker processes.\n\n### workers[].send(data)\n\nMethod for sending data to a specific Worker function.\n\n``` javascript\nworkerPool.workers.forEach((worker) =\u003e worker.send(\"Hello world\"));\n```\n\n### workers[].invoke(fn, data)\n\nMethod for invoking a method on a specific Worker API method.\n\n``` javascript\nworkerPool.workers.forEach((worker) =\u003e worker.invoke(\"say\", \"Hello wold\"));\n```\n\n### workers[].stop()\n\nMethod to stop a specific worker. This will gracefully let the worker process finish any work in progress before stopping it.\n\n``` javascript\nworkerPool.workers.forEach((worker) =\u003e worker.stop());\n```\n\n\n## Worker\n\nClass that allows you to define an API that a worker pool can invoke. All you need to do is extend the Worker class to define your API methods, and export it. Workit will internally manage the lifecycle of your Worker API. The methods exposed by the API get two parameters\n\n- data - data to be processed by the method\n- done - a function to be called when the worker method has finished processing the data. You can alternatively return a Promise. If any data is passed to `done` or the resolved promise, that data will be received by the worker pool for the particular job.\n\nExport a simple Worker API with a method `say` that a worker pool can invoke.\n``` javascript\nimport Workit form 'workit';\n\nclass Worker extends Workit.Worker {\n  say(data) {\n    return Promise.resolve(\"got it\");\n  }\n}\n\nexport default Worker;\n```\n\n## isWorker\n\nFlag that is true if the code reading the value is executing in a worker process. Otherwise the value is not defined. `isWorker` is also available in `process.isWorker`.\n\n\n\n## Thanks!!\n\n\u003e workit used to be taken up in npm by the now defunct project (https://github.com/shannonmoeller/workit). The author - Shannon - of that project is amazing and let me have the name. Thank you, Shannon!\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiguelcastillo%2Fworkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiguelcastillo%2Fworkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiguelcastillo%2Fworkit/lists"}