{"id":15493471,"url":"https://github.com/sinclairzx81/threadbox","last_synced_at":"2025-04-09T13:11:26.897Z","repository":{"id":45524817,"uuid":"245862269","full_name":"sinclairzx81/threadbox","owner":"sinclairzx81","description":"Recursive Worker Threads in NodeJS","archived":false,"fork":false,"pushed_at":"2022-07-14T08:20:30.000Z","size":5344,"stargazers_count":229,"open_issues_count":1,"forks_count":9,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-10-19T11:12:20.724Z","etag":null,"topics":["atomics","channels","nodejs","parallelism","shared-array-buffer","worker-threads"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sinclairzx81.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"license","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-08T17:58:56.000Z","updated_at":"2024-09-04T10:35:13.000Z","dependencies_parsed_at":"2022-09-12T15:22:52.422Z","dependency_job_id":null,"html_url":"https://github.com/sinclairzx81/threadbox","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sinclairzx81%2Fthreadbox","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sinclairzx81%2Fthreadbox/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sinclairzx81%2Fthreadbox/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sinclairzx81%2Fthreadbox/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sinclairzx81","download_url":"https://codeload.github.com/sinclairzx81/threadbox/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248045266,"owners_count":21038555,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atomics","channels","nodejs","parallelism","shared-array-buffer","worker-threads"],"created_at":"2024-10-02T08:07:00.523Z","updated_at":"2025-04-09T13:11:26.881Z","avatar_url":"https://github.com/sinclairzx81.png","language":"TypeScript","readme":"\u003cdiv align='center'\u003e\n\n\u003ch1\u003eThreadBox\u003c/h1\u003e\n\n\u003cp\u003eRecursive Worker Threads in NodeJS\u003c/p\u003e\n\n[![npm version](https://badge.fury.io/js/%40sinclair%2Fthreadbox.svg)](https://badge.fury.io/js/%40sinclair%2Fthreadbox)\n[![Build Status](https://travis-ci.org/sinclairzx81/threadbox.svg?branch=master)](https://travis-ci.org/sinclairzx81/threadbox)\n\n\u003cimg src='./.build/threadbox.png'\u003e\u003c/img\u003e\n\n\u003c/div\u003e\n\n### Example\n\nThe following replicates the above worker graph.\n\n```typescript\nimport { Thread, Sender, Receiver } from '@sinclair/threadbox'\n\nconst WorkerC = Thread.Worker(class {\n  run() {\n    return Math.random()\n  }\n})\n\nconst WorkerB = Thread.Worker(class {\n  async run(sender: Sender) {\n    const c_0 = Thread.Spawn(WorkerC)\n    const c_1 = Thread.Spawn(WorkerC)\n    const c_2 = Thread.Spawn(WorkerC)\n    const c_3 = Thread.Spawn(WorkerC)\n    const [a, b, c, d] = await Promise.all([\n      c_0.run(),\n      c_1.run(),\n      c_2.run(),\n      c_3.run(),\n    ])\n    await sender.send([a, b, c, d])\n    await sender.end()\n    await c_0.dispose()\n    await c_1.dispose()\n    await c_2.dispose()\n    await c_3.dispose()\n  }\n})\nconst WorkerA = Thread.Worker(class {\n  async run(receiver: Receiver) {\n    for await(const [a, b, c, d] of receiver) { }\n  }\n})\n\n// start here ...\nThread.Main(() =\u003e {\n  const [sender, receiver] = Thread.Channel()\n  const a = Thread.Spawn(WorkerA)\n  const b = Thread.Spawn(WorkerB)\n  await Promise.all([\n    a.run(receiver),\n    b.run(sender) \n  ])\n  await a.dispose()\n  await b.dispose()\n})\n```\n\n\u003ca name=\"Overview\"\u003e\u003c/a\u003e\n\n## Overview\n\nThreadBox is a threading library for JavaScript built on top of NodeJS `worker_threads`. It is written to allow for compute intensive and potentially blocking JavaScript routines to be easily executed in remote worker threads. ThreadBox uses a recursive threading model, where spawned threads are created by re-running the applications entry module (typically `app.js`). This approach allows for ergonomic threading, but requires code executed in the global scope to be moved into functions and classes.\n\nThis project is written as a research project to explore the potential for recursive threading in Node. It is offered to anyone who may find it of use.\n\nLicence MIT\n\n\u003ca name=\"Install\"\u003e\u003c/a\u003e\n\n## Install\n\n```bash\n$ npm install @sinclair/threadbox --save\n```\n\n## Contents\n- [Install](#Install)\n- [Overview](#Overview)\n- [Main](#Main)\n- [Worker](#Worker)\n- [Marshal](#Marshal)\n- [Spawn](#Spawn)\n- [Channel](#Channel)\n- [Mutex](#Mutex)\n\n\u003ca name=\"Main\"\u003e\u003c/a\u003e\n\n## Main\n\nUse `Thread.Main(...)` to define the application entry point. This function will only be called once when the process starts, and ignored for subsequent threads.\n\n```typescript\nimport { Thread } from '@sinclair/threadbox'\n\nThread.Main(() =\u003e {\n  \n  console.log('Hello World')\n  \n})\n```\n\n\u003ca name=\"Worker\"\u003e\u003c/a\u003e\n\n## Worker\n\nUse `Thread.Worker(...)` to denote a class as threadable. This enables the class to be spawned via `Thread.Spawn(...)`. The return type of this function returns the inner constructor that can be instanced in the current thread.\n\n```typescript\nimport { Thread } from '@sinclair/threadbox'\n\nconst Basic = Thread.Worker(class {\n    add(a: number, b: number) {\n        return a + b\n    }\n    dispose() { \n        console.log('disposed!')\n    }\n})\n\nThread.Main(async () =\u003e {\n    // instance as thread\n    const thread = Thread.Spawn(Basic)\n    console.log(await thread.add(10, 20))\n    await thread.dispose()\n\n    // instance as local\n    const local = new Basic()\n    console.log(local.add(10, 20))\n})\n```\n\n\u003ca name=\"Spawn\"\u003e\u003c/a\u003e\n\n## Spawn\n\nThe `Thread.Spawn(...)` to spawn a new constructor in a remote worker thread. This function takes the threadable constructor as it's first argument followed by any parameters defined for the constructor.\n\n```typescript\nimport { Thread } from '@sinclair/threadbox'\n\nconst Runner = Thread.Worker(class {\n  constructor(private taskName: string) {\n    console.log(`Runner: ${taskName}`)\n  }\n  process() {\n    console.log(`Runner: execute: ${taskName}`)\n  }\n  dispose() {\n    console.log(`Runner: dispose ${taskName}`)\n  }\n})\n\nThread.Main(async () =\u003e {\n  const runner = Thread.Spawn(Runner, 'Name of Runner')\n  await runner.process()\n  await runner.dispose()\n})\n```\n\n\u003ca name=\"Channel\"\u003e\u003c/a\u003e\n\n## Channel\n\nUse `Thread.Channel\u003cT\u003e()` to create a messaging channel to communicate between threads.\n\n```typescript\nimport { Thread, Sender, Receiver } from '@sinclair/threadbox'\n\nconst Numbers = Thread.Worker(class {\n  start(sender: Sender\u003cnumber\u003e) {\n    for(let i = 0; i \u003c 1024; i++) {\n        sender.send(i)\n    }\n  }\n})\n\nThread.Main(async () =\u003e {\n  const thread = Thread.Spawn(Numbers)\n  const [sender, receiver] = Thread.Channel\u003cnumber\u003e()\n  thread.start(sender)\n  \n  // await values on receiver\n  for await(const value of receiver) {\n    console.log(value)\n  }\n\n  await thread.dispose()\n})\n```\n\n\u003ca name=\"Marshal\"\u003e\u003c/a\u003e\n\n## Marshal\n\nUse `Thread.Marshal(...)` to denote a constructor should be marshalled across threads. This enables class instances to be transferred to remote threads for remote invocation.\n\n```typescript\nimport { Thread } from '@sinclair/threadbox'\n\nconst Transferrable = Thread.Marshal({\n    method() {\n        console.log('Hello World')\n    }\n})\n\nconst Worker = Thread.Worker({\n    execute(transferable: Transferrable) {\n        transferable.method() // callable\n    }\n}\n\nThread.Main(() =\u003e {\n  const thread = spawn(Worker)\n  const transferable = new Transferrable()\n  await thread.execute(transferable)\n  await thread.dispose()\n})\n```\nNote: There is a serialization cost to marshaling. For performance, only `Marshal` when you need to dynamically move logic in and out of threads.\n\n\u003ca name=\"Mutex\"\u003e\u003c/a\u003e\n\n## Mutex\n\nUse `Thread.Mutex(...)` to create a lock on critical sections. This should only be used when two threads reference the same SharedArrayBuffer.\n\n\n\n```typescript\nimport { Thread, Mutex } from '@sinclair/threadbox'\n\nconst Worker = Thread.Worker(class {\n  constructor(private readaonly mutex: Mutex) {}\n\n  execute(data: Uint8Array, value: number) {\n    this.mutex.lock()\n    data[0] = value\n    data[1] = value\n    data[2] = value\n    data[3] = value\n    this.mutex.unlock()\n  }\n})\n\nThread.Main(async () =\u003e {\n\n  const mutex = Thread.Mutex()\n\n  const threads = [\n    Thread.Spawn(Worker, mutex),\n    Thread.Spawn(Worker, mutex),\n    Thread.Spawn(Worker, mutex),\n    Thread.Spawn(Worker, mutex)\n  ]\n\n  const shared = new Uint8Array(new SharedArrayBuffer(4 * Float32Array.BYTES_PER_ELEMENT))\n\n  await Promise.all([\n    threads[0].execute(shared)\n    threads[1].execute(shared)\n    threads[2].execute(shared)\n    threads[3].execute(shared)\n  ])\n\n  await Promise.all([\n    threads[0].dispose()\n    threads[1].dispose()\n    threads[2].dispose()\n    threads[3].dispose()\n  ])\n})\n```\n","funding_links":[],"categories":["TypeScript"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsinclairzx81%2Fthreadbox","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsinclairzx81%2Fthreadbox","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsinclairzx81%2Fthreadbox/lists"}