{"id":21037588,"url":"https://github.com/faranalytics/nodes","last_synced_at":"2025-07-25T22:35:23.318Z","repository":{"id":246851129,"uuid":"824260827","full_name":"faranalytics/nodes","owner":"faranalytics","description":"Nodes provides a framework for building type-safe data transformation graphs using Node.js streams.","archived":false,"fork":false,"pushed_at":"2025-01-18T22:56:44.000Z","size":211,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-10T00:05:21.783Z","etag":null,"topics":["graph","nodejs","stream"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/@farar/nodes","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/faranalytics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-04T17:53:43.000Z","updated_at":"2025-01-18T22:56:46.000Z","dependencies_parsed_at":"2024-07-20T04:25:28.732Z","dependency_job_id":null,"html_url":"https://github.com/faranalytics/nodes","commit_stats":null,"previous_names":["faranalytics/nodes"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faranalytics%2Fnodes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faranalytics%2Fnodes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faranalytics%2Fnodes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faranalytics%2Fnodes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/faranalytics","download_url":"https://codeload.github.com/faranalytics/nodes/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244981292,"owners_count":20542288,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graph","nodejs","stream"],"created_at":"2024-11-19T13:27:00.946Z","updated_at":"2025-03-22T15:44:58.414Z","avatar_url":"https://github.com/faranalytics.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Nodes\n\nNodes provides a framework for building type-safe data transformation graphs using Node.js streams.\n\n## Introduction\n\nNodes provides an intuitive framework for constructing data transformation graphs using native Node.js streams. You can use the built-in library of commonly used data transformation `Node` classes or implement your own.\n\n### Features\n\n- A type-safe graph-like API pattern for building data transformation graphs based on Node.js streams.\n- Consume any native Node.js Readable, Writable, Duplex, or Transform stream and add it to your graph.\n- Error handling and selective termination of inoperable graph components.\n- Automatic message queueing in order to assist with handling of backpressure.\n\n## Table of contents\n\n- [Installation](#installation)\n- [Concepts](#concepts)\n- [Examples](#examples)\n- [API](#api)\n- [How-Tos](#how-tos)\n- [Backpressure](#backpressure)\n- [Best practices](#best-practices)\n- [Error handling](#error-handling)\n- [Versioning](#versioning)\n- [Test](#test)\n- [Support](#support)\n\n## Installation\n\n```bash\nnpm install @farar/nodes\n```\n\n## Concepts\n\n### Node\n\nA `Node` is a component of a graph-like data transformation pipeline. Each `Node` is responsible for transforming its input into an output that can be consumed by its connected `Node` instances. By connecting `Node` instances into a network, sophisticated graph-like data transformation pipelines can be constructed.\n\n## Examples\n\n### _A graph API pattern logger implementation_ \u003csup\u003e\u003csup\u003e\\\u003c/example\\\u003e\u003c/sup\u003e\u003c/sup\u003e\n\nPlease see the [_Streams_ Logger](https://github.com/faranalytics/streams-logger) implementation.\n\n## API\n\n### The Node class\n\n#### new nodes.Node\\\u003cInT, OutT\\\u003e(stream, options)\n\n- `\u003cIntT\u003e` The input into the stream.\n- `\u003cOutT\u003e` The output from the stream.\n- `stream` `\u003cstream.Writable | stream.Readable\u003e` An instance of a `Writable`, `Readable`, `Duplex`, or `Transform` Node.js stream.\n- `options` `\u003cNodeOptions\u003e`\n  - `errorHandler` `\u003c(err: Error, ...params: Array\u003cunknown\u003e) =\u003e void\u003e` An optional error handler that will be used in the event of an internal Error. **Default: `console.error`**\n\n_public_ **node.connect(...nodes)**\n\n- nodes `\u003cArray\u003cT\u003e\u003e` An array of `Node\u003cOutT, unknown\u003e` to be connected to this `Node`.\n\nReturns: `\u003cNode\u003cInT, OutT\u003e\u003e`\n\n_public_ **node.disconnect(...nodes)**\n\n- nodes `\u003cArray\u003cT\u003e\u003e` An array of `Node\u003cOutT, unknown\u003e` to be disconnected from this `Node`.\n\nReturns: `\u003cNode\u003cInT, OutT\u003e\u003e`\n\n_protected_ **node.\\_write(data, encoding)**\n\n- data `\u003cInT\u003e` Data to write to the writable side of the stream.\n- encoding `\u003cNodeJS.BufferEncoding\u003e` An optional Node.js [encoding](https://nodejs.org/api/buffer.html#buffers-and-character-encodings) **Default: `utf-8`**.\n\nReturns: `\u003cPromise\u003cvoid\u003e\u003e`\n\n### The Nodes Config settings object\n\n**Config.errorHandler** `\u003cErrorHandler\u003e` An optional error handler. **Default: `console.error`**\n\n**Config.debug** `\u003cboolean\u003e` Announce internal activities e.g., connectiing and disconnecting from `Node` instances. **Default: `false`**\n\n## How-Tos\n\n### How to implement a data transformation node\n\nIn order to implement a data transformation `Node`, extend the `Node` class and pass a Node.js `stream.Writable` implementation to the super's constructor.\n\nFor example, the following `StringToNumber` implementation will convert a numeric string to a number.\n\n\u003e **NB** In this example, `writableObjectMode` and `readableObjectMode` are both set to `true`; hence, the Node.js stream implementation will handle the input and output as objects. It's important that `writableObjectMode` and `readableObjectMode` accurately reflect the input and output types of your `Node`.\n\n```ts\nimport * as stream from \"node:stream\";\nimport { Config, Node } from \"@farar/nodes\";\n\nexport class StringToNumber extends Node\u003cstring, number\u003e {\n  constructor(options: stream.TransformOptions) {\n    super(\n      new stream.Transform({\n        ...options,\n        ...{\n          writableObjectMode: true,\n          readableObjectMode: true,\n          transform: (\n            chunk: string,\n            encoding: BufferEncoding,\n            callback: stream.TransformCallback\n          ) =\u003e {\n            try {\n              const result = parseFloat(chunk.toString());\n              callback(null, result);\n            } catch (err) {\n              if (err instanceof Error) {\n                callback(err);\n                Config.errorHandler(err);\n              }\n            }\n          },\n        },\n      })\n    );\n  }\n}\n```\n\n### How to consume a Readable, Writable, Duplex, or Transform Node.js stream\n\nIn this hypothetical example a type-safe `Node` is constructed from a `net.Socket`. The resulting `Node` instance can be used in a data transformation graph.\n\n```ts\nimport * as net from \"node:net\";\nimport { once } from \"node:events\";\n\nnet.createServer((socket: net.Socket) =\u003e socket.pipe(socket)).listen(3000);\nconst socket = net.createConnection({ port: 3000 });\nawait once(socket, \"connect\");\nconst socketHandler = new Node\u003cBuffer, Buffer\u003e(socket);\n```\n\n## Backpressure\n\nThe `Node` class has a `_write` method that respects backpressue; when a stream is draining it will queue messages until a `drain` event is emitted by the Node's stream. Your application can optionally monitor the size of the queue and respond appropriately.\n\nIf you have a stream that is backpressuring, you can increase the high water mark on the stream in order to mitigate drain events.\n\n## Best practices\n\n### Avoid reuse of `Node` instances (_unless you know what you are doing!_).\n\nReusing the same `Node` instance can result in unexpected phenomena. If the same `Node` instance is used in different locations in your graph, you need to think carefully about the resulting edges that are connected to both the input and the output of the `Node` instance. Most of the time if you need to use the same class of `Node` more than once, it's advisable to create a new instance for each use.\n\n## Error handling\n\nNodes may be used in diverse contexts, each with unique requirements. Nodes _should_ never throw if the API is used in accordance with the documentation. However, \"_phenomena happens_\"; hence, you may choose to handle errors accordingly!\n\nNodes defaults to logging its errors to `process.stderr`. If your application requires that errors throw, you may set an `errorHandler` on the `Config` object that does that. Alternatively, your handler may consume the `Error` and handle it otherwise.\n\n### Optionally configure all internal errors to be thrown\n\n```ts\nimport { Config } from \"@farar/nodes\";\nConfig.errorHandler = (err: Error) =\u003e {\n  throw err;\n};\n```\n\n## Versioning\n\nThe Nodes package adheres to semantic versioning. Breaking changes to the public API will result in a turn of the major. Minor and patch changes will always be backward compatible.\n\nExcerpted from [Semantic Versioning 2.0.0](https://semver.org/):\n\n\u003e Given a version number MAJOR.MINOR.PATCH, increment the:\n\u003e\n\u003e 1. MAJOR version when you make incompatible API changes\n\u003e 2. MINOR version when you add functionality in a backward compatible manner\n\u003e 3. PATCH version when you make backward compatible bug fixes\n\u003e\n\u003e Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.\n\n## Test\n\n### How to run the test\n\n#### Clone the repository.\n\n```bash\ngit clone https://github.com/faranalytics/nodes.git\n```\n\n#### Change directory into the root of the repository.\n\n```bash\ncd nodes\n```\n\n#### Install dependencies.\n\n```bash\nnpm install \u0026\u0026 npm update\n```\n\n#### Run the tests.\n\n```bash\nnpm test\n```\n\n## Support\n\nIf you have a feature request or run into any issues, feel free to submit an [issue](https://github.com/faranalytics/nodes/issues) or start a [discussion](https://github.com/faranalytics/nodes/discussions). You’re also welcome to reach out directly to one of the authors.\n\n- [Adam Patterson](https://github.com/adamjpatterson)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffaranalytics%2Fnodes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffaranalytics%2Fnodes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffaranalytics%2Fnodes/lists"}