{"id":20852963,"url":"https://github.com/julianpoemp/clusterfy","last_synced_at":"2026-04-27T02:32:52.972Z","repository":{"id":123839827,"uuid":"608035433","full_name":"julianpoemp/clusterfy","owner":"julianpoemp","description":"NodeJS package that simplifies process management and Inter Process Communication in NodeJS cluster.","archived":false,"fork":false,"pushed_at":"2024-02-18T14:54:50.000Z","size":2491,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-26T23:53:58.607Z","etag":null,"topics":["cluster","inter-process-communication","javascript-library","nodejs","shared-memory-communication","typescript-library"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/julianpoemp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-01T07:20:40.000Z","updated_at":"2023-11-21T14:59:26.000Z","dependencies_parsed_at":null,"dependency_job_id":"63c7488b-588a-4801-bddf-27acceb612e7","html_url":"https://github.com/julianpoemp/clusterfy","commit_stats":{"total_commits":33,"total_committers":1,"mean_commits":33.0,"dds":0.0,"last_synced_commit":"1b3dec4397923c4d47b9e7d63ec645a2749b9523"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/julianpoemp/clusterfy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/julianpoemp%2Fclusterfy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/julianpoemp%2Fclusterfy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/julianpoemp%2Fclusterfy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/julianpoemp%2Fclusterfy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/julianpoemp","download_url":"https://codeload.github.com/julianpoemp/clusterfy/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/julianpoemp%2Fclusterfy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32320383,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-26T23:26:28.701Z","status":"online","status_checked_at":"2026-04-27T02:00:06.769Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster","inter-process-communication","javascript-library","nodejs","shared-memory-communication","typescript-library"],"created_at":"2024-11-18T03:19:23.207Z","updated_at":"2026-04-27T02:32:52.950Z","avatar_url":"https://github.com/julianpoemp.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"![ClusterfyLogo](https://github.com/julianpoemp/clusterfy/assets/25926588/a67d8387-366f-4143-a8f3-ddf3087f98f4)\n\n\u003cp style=\"text-align:center;\"\u003e\n    \n\u003ca href=\"https://www.npmjs.com/package/clusterfy\"\u003e\u003cimg alt=\"npm\" src=\"https://img.shields.io/npm/v/clusterfy\"\u003e\u003c/a\u003e\n\n\n\nThis library simplifies managing processes in a NodeJS cluster. It supports IPC via messaging and a shared storage\nmanaged by the primary process.\n\n\u003cimg src=\"https://raw.githubusercontent.com/julianpoemp/clusterfy/main/docs/images/TerminalDemo.gif\" alt=\"terminal demo gif\"/\u003e\n\nFor presentation timeouts were inserted. You can find the code to this terminal\npreview [here](https://github.com/julianpoemp/clusterfy/blob/main/packages/demo/src/main.ts).\n\n\u003c/p\u003e\n\n# Features\n\n- Simple API for creating, watching and controlling workers in a cluster.\n- Shared storage managed by primary process. Workers can set and retrieve this storage.\n- Simple API for sending commands between primary and workers: primary \u003c-\u003e worker or worker \u003c-\u003e worker\n- Easy way to add custom commands\n- Simple event handling using RxJS (e.g. using Observable)\n- Extended workers with more attributes like name and status\n- Graceful shutdown on process signals\n- ES Module with Typescript definitions\n\n# Applications\n\nYou can use Clusterfy in all NodeJS applications with Node \u003e= 16 and servers like ExpressJS or NestJS.\n\n# How it works\n\nThe primary process is the manager who knows everything about the workers and controls these. Furthermore the primary\nprocess manages a shared storage (JS object). If worker want to retrieve or save values to the storage they send\nrequests to the primary process which returns or saves the value.\n\nCommands from primary process to worker process and vice versa are sent via IPC as ClusterfyIPCEvent objects.\nIt's also possible to send commands from one worker to another. This works thanks to the primary process that\nredirects the messages from one worker to another. With this construct you can send commands asynchronously between\ndifferent processes.\n\n# Installation\n\n````\nnpm install clusterfy --save\n````\n\nFurthermore, you need to install rxjs \u003e= 7.5.0 and uuid \u003e= 7.0.3 if not already installed.\n\n# Usage\n\nClusterfy is a static class that can be called everywhere. At the entrypoint of your application you want to start the\ncluster. So you need to initiate the primary process and its workers like that:\n\n````Typescript\nimport {Clusterfy, ClusterfyIPCEvent, ClusterfyStorage} from 'clusterfy';\n\nconst onShutdown = async () =\u003e {\n    console.log(`Simulate cleanup on shutdown for ${Clusterfy.currentLabel}...`);\n\n    let j = 0;\n    for (let i = 0; i \u003c 100000; i++) {\n        j++;\n    }\n    console.log(`All cleaned up for ${Clusterfy.currentLabel} OK`);\n};\n\nasync function main() {\n    if (Clusterfy.isCurrentProcessPrimary()) {\n        // initStorage shared memory\n        const sharedMemory = new ClusterfyStorage({\n            test: {\n                some: 1,\n            },\n        });\n        Clusterfy.initStorage(sharedMemory);\n\n        // create some workers asynchronously\n        Clusterfy.fork('SomeName1');\n        Clusterfy.fork();\n        // now create worker that is revived automatically after it died\n        Clusterfy.fork('EmailServer', {revive: true});\n        await Clusterfy.initAsPrimary({\n            gracefulOnSignals: ['SIGINT', 'SIGTERM'],\n        });\n        Clusterfy.registerShutdownMethod('default', onShutdown);\n\n        // now you can use Clusterfy on primary as you like\n        Clusterfy.events.subscribe({\n            next: (event: ClusterfyIPCEvent) =\u003e {\n                console.log(`Primary got event ${event.type} from worker ${event.senderID}`);\n            }\n        });\n    } else {\n        await Clusterfy.initAsWorker({\n            gracefulOnSignals: ['SIGINT', 'SIGTERM'],\n        });\n        Clusterfy.registerShutdownMethod('default', onShutdown);\n        // worker retrieved metadata from primary on initialization\n        console.log(`Worker ${Clusterfy.currentWorker.name} is ready.`);\n\n        // now you can use Clusterfy on worker as you like\n\n        Clusterfy.events.subscribe({\n            next: (event: ClusterfyIPCEvent) =\u003e {\n                console.log(`Worker ${Clusterfy.currentWorker.name} got event ${event.type} from worker ${event.senderID}`);\n            }\n        });\n    }\n}\n\nmain();\n\n````\n\n## Getter\n\n### currentWorker(): ClusterfyWorker\n\nReturns current worker (only on worker). Returns undefined else.\n\n### currentLabel(): string\n\nReturns \"Primary\" on primary or \"Workername (id)\" on worker.\n\n### storage(): ClusterfyStorage\u003cany\u003e\n\nReturns shared storage (only on primary). Returns undefined else.\n\n### events(): Subject\u0026lt;ClusterfyIPCEvent\u0026gt;\n\nReturns observable of all events to this worker or primary.\n\n### workers(): ClusterfyWorker[]\n\nReturns array of workers (only on primary). Returns empty array else.\n\n## Functions\n\n### initStorage\u0026lt;T\u0026gt;(storage: ClusterfyStorage\u003cT\u003e)\n\nInitializes the storage. Call this method on primary. The storage must be an object, e.g.\n\n````\n{\n    test: {\n        something: 123\n    }\n}\n````\n\n### fork(name?: string, options?: ClusterfyWorkerOptions)\n\nCreates a new worker with given name and options.\n\n#### Options\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003cb\u003erevive\u003c/b\u003e\u003c/td\u003e\n\u003ctd\u003eIf the worker dies because of an error it will be revived.\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n### async initAsPrimary(shutdownOptions?: ClusterfyShutdownOptions)\n\nInitializes the primary with Clusterfy. This method must be called on primary (see example). If you want to include\ngraceful shutdown on process signals you need to add `shutdownOptions`. Resolves as soon as all workers are ready.\n\n### async initAsWorker(shutdownOptions?: ClusterfyShutdownOptions)\n\nInitializes the worker with Clusterfy and waits for metadata from primary. This method must be called on worker (see\nexample). Wait until this method returns. If you want to include graceful shutdown on process signals you need to\nadd `shutdownOptions`. Resolves as soon as worker is ready.\n\n### registerShutdownMethod(name: string, command: (signal: NodeJS.Signals) =\u003e Promise\u003cvoid\u003e)\n\nRegisters a new method that is run on shutdown.\n\n### removeShutdownMethod(name: string)\n\nRemoves an existing shutdown method.\n\n### async saveToStorage(path: string, value: any)\n\nSaves a serializable value to a path in the shared storage on primary. Path should be a string with dot notation to the\nattribute , e.g. \"test.something\". This method returns as soon as saved.\n\n### async retrieveFromStorage\u0026lt;T\u0026gt;(path: string)\n\nReturns a serializable value from a given path in shared storage. Path should be a string with dot notation to the\nattribute , e.g. \"test.something\". This method returns the value of type `T` as soon as retrieved.\n\n### async shutdownWorker(worker: ClusterfyWorker, timeout = 2000)\n\nIf you call `Clusterfy.shutdownWorker(worker, 2000)` from primary it sends a cy_shutdown command to a worker. The worker\ngets status \"STOPPING\" and should exit itself using `process.exit(0)` in 2 seconds (graceful shutdown). If the worker\ndoesn't exit itself,\nthe primary kill it.\n\nThat means: the algorithm you are using for processing in a worker should check if the status is \"\nSTOPPING\" using `Clusterfy.currentWorker.status` and then exit itself. The algorithm should also change the status to \"\nPROCESSING\" after it started processing\nand change it\nback to \"IDLE\" after finished. If a shutdown is received Clusterfy checks if the status is \"IDLE\" and\ncalls `process.exit(0)` or changes the status to \"STOPPING\".\n\n### getStatistics()\n\nReturns a object with statistics about the workers. Call it on primary.\n\n### outputStatisticsTable()\n\nOutputs statistics from getStatistics to console. Call it on primary.\n\n### registerIPCCommand(command: clusterfyCommand)\n\nRegisters a new custom command to the list of supported commands. Call this method on worker and primary.\n\n### changeCurrentWorkerStatus(status: ClusterfyWorkerStatus)\n\nChanges the status of the current worker and emits event of type \"status\" to itself and to primary.\n\n## Create custom command\n\nEach command should extend class `ClusterfyCommand`. See examples\nin [commands](https://github.com/julianpoemp/clusterfy/tree/main/packages/clusterfy/src/lib/commands).\nCall `registerIPCCommand(command)` on primary and worker.\n\n````Typescript\nexport class ClusterfyCommand {\n    name: string;\n    target: 'primary' | 'worker';\n    runOnTarget: (\n        args: Record\u003cstring, any\u003e,\n        commandEvent?: ClusterfyCommandRequest\u003cany\u003e\n    ) =\u003e Promise\u003cClusterfyCommandRequestResult\u003cany\u003e\u003e;\n\n    constructor(options?: Partial\u003cClusterfyCommand\u003e) {\n        if (options) {\n            Object.assign(this, options);\n        }\n    }\n}\n````\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003cb\u003ename\u003c/b\u003e\u003c/td\u003e\n\u003ctd\u003eThe name of the command in lower case. It should not start with \"cy_\" because only commands from Clusterfy should be named like that.\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003cb\u003etarget\u003c/b\u003e\u003c/td\u003e\n\u003ctd\u003eTake \"primary\" if this command should be run by primary and \"worker\" otherwise.\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003cb\u003erunOnTarget\u003c/b\u003e\u003c/td\u003e\n\u003ctd\u003emethod that is run on the target.\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n# Development\n\n1. Install dependencies\n\n````\n   npm install\n````\n\n2. Start in one terminal\n\n````\n   npm run watch:lib\n````\n\n3. Run demo to test it.\n\n````\n   npm run start:demo\n ````\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjulianpoemp%2Fclusterfy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjulianpoemp%2Fclusterfy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjulianpoemp%2Fclusterfy/lists"}