https://github.com/smoren/multiprocessor-ts
Multiprocessing pool for NodeJS and TypeScript
https://github.com/smoren/multiprocessor-ts
child-process child-process-pool map map-unordered multicore multicore-cpu multiprocess multiprocessing parallel pool worker workers
Last synced: about 12 hours ago
JSON representation
Multiprocessing pool for NodeJS and TypeScript
- Host: GitHub
- URL: https://github.com/smoren/multiprocessor-ts
- Owner: Smoren
- License: mit
- Created: 2025-01-04T14:20:27.000Z (4 months ago)
- Default Branch: master
- Last Pushed: 2025-01-21T16:17:21.000Z (3 months ago)
- Last Synced: 2025-04-28T15:15:42.595Z (about 12 hours ago)
- Topics: child-process, child-process-pool, map, map-unordered, multicore, multicore-cpu, multiprocess, multiprocessing, parallel, pool, worker, workers
- Language: TypeScript
- Homepage: https://www.npmjs.com/package/multiprocessor
- Size: 23.4 KB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Multiprocessing Pool Implementation for TypeScript
[](https://www.npmjs.com/package/multiprocessor)
[](https://www.npmjs.com/package/multiprocessor)
[](https://coveralls.io/github/Smoren/multiprocessor-ts?branch=master)
[](https://bundlephobia.com/result?p=multiprocessor)
[](https://opensource.org/licenses/MIT)
Description
-----------
Multiprocessing pool implementation for NodeJS and TypeScript.
Real multiprocessing is implemented using [child_process](https://nodejs.org/api/child_process.html) module.
Setup
-----
```bash
npm i multiprocessor
```
Usage example
-------------
```typescript
import { Pool } from 'multiprocessor';
const poolSize = 4;
const pool = new Pool(poolSize);
const input = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const result = await pool.map(input, calcSinTask, {
onTaskSuccess: (result: number, input: number, index: number) => {
console.log(`Task #${index} | result: ${result}, input: ${input}`);
},
onTaskError: (error: string, input: number, index: number) => {
console.log(`Task #${index} | error: ${error}, input: ${input}`);
}
});
pool.close();
console.log(result);
// [ 0.8414, 0.9092, 0.1411, ... ]
function calcSinTask(x: number): number {
let result = 0;
let sign = 1;
let power = x;
let factorial = 1;
for (let n = 0; n < 1000000; n++) {
if (n > 0) {
factorial *= (2 * n) * (2 * n + 1);
power *= x * x;
sign *= -1;
}
const delta = sign * (power / factorial);
if (isNaN(result + delta)) {
return result
}
result += delta;
}
return result;
}
```
### Example with import
You can run this example from [this repository](https://github.com/Smoren/multiprocessor-example-ts).
```typescript
// File: src/index.ts
import { Pool } from 'multiprocessor';
const poolSize = 4;
const pool = new Pool(poolSize);
const input = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const result = await pool.map(input, calcSinTask, {
onTaskSuccess: (result: number, input: number, index: number) => {
console.log(`Task #${index} | result: ${result}, input: ${input}`);
},
onTaskError: (error: string, input: number, index: number) => {
console.log(`Task #${index} | error: ${error}, input: ${input}`);
}
});
pool.close();
console.log(result);
// [ 0.8414, 0.9092, 0.1411, ... ]
async function calcSinTask(x: number): Promise {
const dirName = __dirname.replace('/node_modules/multiprocessor/lib', '/src');
const { calcSin } = await import(`${dirName}/path/to/your/module`);
return calcSin(x);
}
```
```typescript
// File: src/path/to/your/module.ts
export function calcSin(x: number): number {
let result = 0;
let sign = 1;
let power = x;
let factorial = 1;
for (let n = 0; n < 1000000; n++) {
if (n > 0) {
factorial *= (2 * n) * (2 * n + 1);
power *= x * x;
sign *= -1;
}
const delta = calcDelta(sign, power, factorial);
if (isNaN(result + delta)) {
return result
}
result += delta;
}
return result;
}
function calcDelta(sign: number, power: number, factorial: number): number {
return sign * (power / factorial);
}
```
API Reference
-------------
For detailed documentation and usage examples, please refer to the [API documentation](https://smoren.github.io/multiprocessor-ts/).
## Types
```typescript
export type Task = (input: TInput) => (Promise | TResult);
export type TaskSuccessHandler = (result: TResult, input: TInput, index: number) => void;
export type TaskErrorHandler = (error: string, input: TInput, index: number) => void;
export type TaskResponse = [number, TResult | undefined, string | undefined];
```
## Pool
```typescript
class Pool extends EventEmitter {
/**
* Create a new pool with the specified number of workers.
*
* @param poolSize The number of workers to create in the pool.
*/
constructor(poolSize: number);
/**
* Asynchronously processes tasks from the provided inputs in an ordered manner.
* Tasks are executed concurrently using a pool of workers.
*
* @template TInput The type of the input elements.
* @template TResult The type of the result elements.
*
* @param inputs An iterable or async iterable of input elements.
* @param task The task to execute for each input element.
* @param taskHandlers Optional handlers for task events (onTaskSuccess, onTaskError).
*
* @returns A promise that resolves to an array of task results in the order of the input elements.
*/
public async map(
inputs: Iterable | AsyncIterable,
task: Task,
taskHandlers?: TaskHandlers,
): Promise>;
/**
* Asynchronously processes tasks from the provided inputs in a lazy ordered manner.
* Tasks are executed concurrently using a pool of workers.
*
* @template TInput The type of the input elements.
* @template TResult The type of the result elements.
*
* @param inputs An iterable or async iterable of input elements.
* @param task The task to execute for each input element.
* @param taskHandlers Optional handlers for task events (onTaskSuccess, onTaskError).
*
* @returns An async generator yielding results of the tasks in the order of the input elements.
*/
public async *imap(
inputs: Iterable | AsyncIterable,
task: Task,
taskHandlers?: TaskHandlers,
): AsyncGenerator;
/**
* Asynchronously processes tasks from the provided inputs in a lazy unordered manner.
* Tasks are executed concurrently using a pool of workers.
*
* @template TInput The type of the input elements.
* @template TResult The type of the result elements.
*
* @param inputs An iterable or async iterable of input elements.
* @param task The task to execute for each input element.
* @param taskHandlers Optional handlers for task events (onTaskSuccess, onTaskError).
*
* @returns An async generator yielding results of the tasks in completion order.
*/
public async *imapUnordered(
inputs: Iterable | AsyncIterable,
task: Task,
taskHandlers?: TaskHandlers,
): AsyncGenerator;
/**
* Asynchronously processes tasks from the provided inputs in a lazy unordered manner with extended information.
* Tasks are executed concurrently using a pool of workers.
*
* @template TInput The type of the input elements.
* @template TResult The type of the result elements.
*
* @param inputs An iterable or async iterable of input elements.
* @param task The task to execute for each input element.
* @param taskHandlers Optional handlers for task events (onTaskSuccess, onTaskError).
*
* @returns An async generator yielding task responses containing the index, result or error for each task.
*/
public async *imapUnorderedExtended(
inputs: Iterable | AsyncIterable,
task: Task,
taskHandlers?: TaskHandlers,
): AsyncGenerator>;
/**
* Closes the worker pool by terminating all worker processes.
* This method should be called when the pool is no longer needed
* to ensure that all resources are properly released.
*/
public close();
}
```
Unit testing
------------
```bash
npm i
npm run test
```
License
-------
Multiprocessor TS is licensed under the MIT License.