{"id":22407560,"url":"https://github.com/f5io/pool","last_synced_at":"2025-07-26T13:41:47.551Z","repository":{"id":42801005,"uuid":"102282941","full_name":"f5io/pool","owner":"f5io","description":"A highly flexible process pooling library for Node.js","archived":false,"fork":false,"pushed_at":"2022-03-26T21:32:01.000Z","size":437,"stargazers_count":19,"open_issues_count":9,"forks_count":1,"subscribers_count":11,"default_branch":"master","last_synced_at":"2023-08-01T05:59:10.601Z","etag":null,"topics":["concurrency","csp","nodejs","pooling"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/f5io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-09-03T17:47:06.000Z","updated_at":"2023-07-27T23:38:33.000Z","dependencies_parsed_at":"2022-08-23T01:41:19.160Z","dependency_job_id":null,"html_url":"https://github.com/f5io/pool","commit_stats":null,"previous_names":[],"tags_count":11,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/f5io%2Fpool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/f5io%2Fpool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/f5io%2Fpool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/f5io%2Fpool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/f5io","download_url":"https://codeload.github.com/f5io/pool/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228283750,"owners_count":17896267,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concurrency","csp","nodejs","pooling"],"created_at":"2024-12-05T11:14:45.770Z","updated_at":"2024-12-05T11:14:46.565Z","avatar_url":"https://github.com/f5io.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# @f5io/pool\n\nA highly flexible process pooling library for Node.js. Built with [@f5io/csp](https://github.com/f5io/csp).\n\n[![npm version](https://badge.fury.io/js/%40f5io%2Fpool.svg)](https://badge.fury.io/js/%40f5io%2Fpool)\n\n## Installation\n\n```\n$ npm install --save @f5io/pool\n```\n\nor\n\n```\n$ yarn add @f5io/pool\n```\n\n## API\n\nThis library exposes a single factory method for creating pools.\n\n### `createPool({ poolSize = 5, createProcess, createAsyncProcess, handler })` -\u003e `Pool|Promise\u003cPool\u003e`\n\nThe pool factory takes an options object containing 3 of 4 properties:\n\n- `poolSize` - defaults to 5, determines the size of the pool\n- `createProcess` - defines a `process` factory function which can return anything\n- `createAsyncProcess` - defines an async `process` factory which can return anything, useful if your process requires time to become active.\n- `handler(process, input)` -\u003e `Promise` - defines a function which handles a unit of work. The handler must return a `Promise` and receives a `process` (as defined by the `process` factory) and the `input` from a call to `run` on the pool\n\nYou must supply only one of `createProcess` or `createAsyncProcess`! If you supply `createAsyncProcess` the return value of the `createPool` factory will be a `Promise\u003cPool\u003e`.\n\nA returned `Pool` exposes 2 methods:\n\n- `Pool.run(input)` -\u003e `Promise` - The interface defined to run against the pool of processes, supplied input can be of any type as the handler supplied at `pool` creation defines how the input interacts which the underlying process\n- `Pool.close` -\u003e `Promise` - A mechanism for destroying the `pool` when it is no longer needed\n\n## Example Usage\n\n### Network I/O parallelisation\n\nBy defining our process as a plain `Symbol`, or `true` for that matter, we can limit request parallelisation to the size of the pool.\n\n```javascript\nconst assert = require('assert');\nconst fetch = require('node-fetch');\nconst createPool = require('@f5io/pool');\n\nconst { run, close } = createPool({\n  poolSize: 2,\n  createProcess: () =\u003e Symbol('process'),\n  handler: (_, query) =\u003e {\n    console.log(`🚀  running request with query: ${query}`);\n    return fetch(`https://postman-echo.com/get?q=${query}`)\n      .then(res =\u003e res.json())\n      .then(res =\u003e assert.equal(res.args.q, query));\n      .then(_ =\u003e console.log(`👌  request completed successfully`));\n  }\n});\n\n(async () =\u003e {\n  const queries = Array.from({ length: 20 }, (_, i) =\u003e run(`${++i}`));\n  await Promise.all(queries);\n  close();\n})();\n```\n\n![request parallelisation](/assets/pool.request.gif?raw=true)\n\n### Child process pooling\n\nFor spawning multiple child processes and spreading work across processes in the pool.\n\n```javascript\nconst assert = require('assert');\nconst { spawn } = require('child_process');\nconst createPool = require('@f5io/pool');\n\nconst { run, close } = createPool({\n  poolSize: 10,\n  createProcess: () =\u003e {\n    const p = spawn('cat', [ '-' ]);\n    p.stdin.setEncoding('utf-8');\n    p.stdout.setEncoding('utf-8'); \n    return p;\n  },\n  handler: (p, input) =\u003e\n    new Promise(resolve =\u003e {\n      p.stdout.once('data', d =\u003e {\n        assert(d, input);\n        console.log(`👌  received data: ${d.trim()} from pid: ${p.pid}`);\n        resolve(d);\n      });\n      console.log(`🚀  sending data: ${input.trim()} to pid: ${p.pid}`);\n      p.stdin.write(input);\n    }),   \n});\n\n(async () =\u003e {\n  const inputs = Array.from({ length: 100 }, (_, i) =\u003e run(`${++i}\\n`));\n  await Promise.all(inputs);\n  close();\n})();\n```\n\n![child process pool](/assets/pool.spawn.gif?raw=true)\n\n## Contributions\n\nContributions are welcomed and appreciated!\n\n1. Fork this repository.\n1. Make your changes, documenting your new code with comments.\n1. Submit a pull request with a sane commit message.\n\nFeel free to get in touch if you have any questions.\n\n## License\n\nPlease see the `LICENSE` file for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ff5io%2Fpool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ff5io%2Fpool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ff5io%2Fpool/lists"}