{"id":15437123,"url":"https://github.com/awwright/node-simplex-pair","last_synced_at":"2026-05-19T15:38:09.903Z","repository":{"id":139430621,"uuid":"280974726","full_name":"awwright/node-simplex-pair","owner":"awwright","description":"A proposal for better streams in Node.js","archived":false,"fork":false,"pushed_at":"2020-07-20T00:23:43.000Z","size":10,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-10-07T17:00:00.075Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/awwright.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-20T00:21:17.000Z","updated_at":"2020-07-20T00:23:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"3eea082a-9882-4269-96d3-1601daa134d5","html_url":"https://github.com/awwright/node-simplex-pair","commit_stats":{"total_commits":1,"total_committers":1,"mean_commits":1.0,"dds":0.0,"last_synced_commit":"2ad5af682304bc9ff3bea47c519bb4ac95cadc6d"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awwright%2Fnode-simplex-pair","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awwright%2Fnode-simplex-pair/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awwright%2Fnode-simplex-pair/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awwright%2Fnode-simplex-pair/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/awwright","download_url":"https://codeload.github.com/awwright/node-simplex-pair/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244583196,"owners_count":20476233,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T18:55:13.185Z","updated_at":"2026-05-19T15:37:59.889Z","avatar_url":"https://github.com/awwright.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Node.js Simplex Pair Proposal\n\nThis is a proposal for improving the modularity and simplicity of streams. It intends to be not-incompatible with previous stream interfaces.\n\nThere are multiple goals, many of which may be implemented independent of the others:\n\n* Publish `SimplexPair` and `DuplexPair` utilities, which allow streams to be created with a symmetric API\n* reduce the number of locations where data is buffered, for performance and to avoid \"bufferbloat\" situations\n* create a symmetrical interface for producing or consuming streaming data (e.g. use `write` calls instead of `push`)\n* standardizing the order of events, to make evaluation order more predictable\n* to expose some events as promises, to provide better guarentees about runtime order\n\n## Interface Source\n\n```\ninterface Source {\n\tEvent: \"readable\"\n\tEvent: \"end\"\n\tEvent: \"error\"\n\tboolean readable;\n\tfunction read([n]);\n\tfunction close();\n\tfunction pipe(dst);\n\tPromise final;\n\tstatic function [Symbol.hasInstance]();\n\tfunction [Symbol.asyncIterator]();\n}\n```\n\nThis is the basic interface by which data on a stream is consumed.\n\nThe data that is read typically comes from one of two sources:\n\n* The data is generated as it is read, as fast as the application can read it.\n* The data is read from a buffer, which is filled from a separate Sink interface (see `SimplexPair` below).\n\nSeveral different mechanisms may be used to work with the available data:\n\n* `Source#on(\"readable\")` and `Source#read()`\n* `Source#pipe(dst)`\n* `for await (const segment of source){ ... }` via the async iterator,\n* `Source#on(\"data\")` or `Source#resume()`, if the Flowing API is implemented\n\n\n### Source#on(\"readable\")\n\nThe \"readable\" event signals that there is data on the stream, which may be read using `Source#read`\n\n\n### Source#pipe(dst)\n\nAny data that becomes available is immediately written to the Sink `dst`.\n\n\n### Source#readable\n\nTrue if `Source#read` will return data if called. This is set to `true` when \"readable\" is emitted, and `false` if `Source#read()` returns `null`.\n\n\n### Source#read([n])\n\nReturns up to _n_ units of data from the stream. If there is no additional data to be read, this will return `null`, and no more data will be available until the next \"readable\" event.\n\n\n### Source#close()\n\nIndicates the application does not intend to consume any more data; this signals the Sink side to stop writing.\n\n\n### Source#on(\"error\")\n\nThe \"error\" event signals an error\n\n\n### Source#on(\"end\")\n\nThe \"end\" event signals an EOF event, that no more data will be available.\n\n\n### Source#final\n\nA Promise that resolves when the stream will have no more data available (an EOF), or rejects when no more data will be available due to an error event.\n\nReading this property will add an \"error\" listener, or otherwise turn off fatal errors when no listeners are attached to \"error\".\n\n\n### `Source[Symbol.hasInstance] ()`\n\nReturns `true` for `Source` or `Duplex`, to implement `new Duplex instanceof Source`.\n\n### `Source#[Symbol.asyncIterator]()`\n\nImplements the async iterator pattern, that allows for the following:\n\n```javascript\nasync function print(readable) {\n  readable.setEncoding('utf8');\n  let data = '';\n  for await (const chunk of readable) {\n    data += chunk;\n  }\n  console.log(data);\n}\n```\n\n\n### Flowing API\n\n`Source#pause`, `Source#resume`, `Source#isPaused`, and `Source#on(\"data\")` may also be implemented for backwards compatibility with Streams 1. In this case, the stream starts \"paused\".\n\n\n## Interface Sink\n\n```\ninterface Sink {\n\tEvent: \"drain\"\n\tEvent: \"close\"\n\tfunction write(segment)\n\tfunction end(segment)\n\tfunction cork()\n\tfunction uncork()\n\tstatic function [Symbol.hasInstance]();\n}\n```\n\nThis is the primary interface by which data is written to a stream. The data is typically stored in a buffer to be available for reading via a Source interface, or delivered to the operating system kernel.\n\n\n### Sink#write(segment)\n\nThis call writes data to the stream.\n\nIf this function returns `false`, it indicates the stream has become congested and the application should stop writing to it. If the writes are processed from an upstream source, the application should stop reading from that source too. This is the primary mechanism used to control how fast data is sent through the stream.\n\n\n### Sink#end([segment])\n\nThis call writes the given segment to the stream, if provided; then closes the stream.\n\n\n### Sink#on(\"drain\")\n\nThis event indicates that all the buffered data has been read, and the application should resume writing, if it has paused.\n\n\n### Sink#on(\"close\")\n\nIndicates that the Source side has stopped reading the stream, and will not read any more data. Any further data written will create an error.\n\n\n### Sink#cork([fn])\n\nData written to the Sink will be buffered, and not become readable until `Sink#uncork()` is called.\n\nThis feature is typically used for optimizations, so that Sink#write may be called in succession with lots of small pieces of data, without being sent as separate packets over the network.\n\nIf `fn` is provided, it will be called in-line, and the stream automatically uncork when the function returns, or if it returns a Promise, when the promise resolves.\n\n\n### `Sink[Symbol.hasInstance]()`\n\nReturns `true` for `Sink` or `Duplex`, to implement `new Duplex instanceof Sink`.\n\n\n## Interface Duplex\n\nA Duplex is a stream that is both readable and writable: You can write data to it, and/or read data from it.\n\n```\ninterface Duplex implements Source, Sink {\n\tSource readableSide;\n\tSink writableSide;\n}\n```\n\nIt has two properties `readableSide` and `writableSide` which exposes only that \"half\" of the Duplex stream.\n\nTwo properties `readableSide` and `writableSide` are exposed, which allows you to pass only a Source or Sink stream to another part of the application. This feature allows you to create encapsulated streams, so that a pointer to a readable stream does not grant write access; and vice-versa.\n\nThere are primarily two types of Duplex streams: a PassThrough/SimplexPair, and each side of a DuplexPair.\n\nECMAScript does not have a concept of multiple inheritance, so the prototypes are copied in, and `instanceof` support is implemented via `Sink[Symbol.hasInstance]` and `Source[Symbol.hasInstance]`.\n\n\n## Class SimplexPair (a.k.a. PassThrough)\n\n`SimplexPair` is one of the two primary ways that an application can make data available for reading on a Source interface. It is a special type of Duplex that keeps a modest buffer, which is filled through the Sink interface, and is drained from the Source interface.\n\nIt is essentially the same as a Node.js PassThrough, but with the addition of `writableSide` and `readableSide` properties inherited from the Duplex interface.\n\n```\ninterface SimplexPair : Duplex {\n\tfunction _read(n);\n}\n```\n\nBy using the `readableSide` and `writableSide` properties, these two functions operate the same way:\n\n```javascript\nfunction a(){\n\tconst { writableSide, readableSide } = new SimplexPair();\n\tprocess.nextTick(() =\u003e writableSide.end(\"foo\\r\\n\"));\n\treturn readableSide;\n}\nfunction b(){\n\tconst pair = new SimplexPair();\n\tprocess.nextTick(() =\u003e pair.end(\"foo\\r\\n\"));\n\treturn pair.readableSide;\n}\n```\n\n`SimplexPair` forms the basis of how virtually all `Source` and `Sink` streams are created in userland.\n\n\n### SimplexPair#_read(n)\n\nThis is an optimization around creating a Source stream that does not depend on data becoming available, but instead can be generated as fast as the receiving end can read it.\n\nWhenever the application calls `SimplexPair#read`, and there is insufficient data in the buffer, this will trigger a call to the user-implemented function `_read`; at this time the user may generate the requested number of bytes, and either return them, or make equivalent (but probably suboptimal) calls using `write`, `end`, and/or `destroy` (either directly on the `SimplexPair` instance, or on the `writableSide` property).\n\nBy default, this function will not write or return anything; and so cause the `read()` call to return `null`.\n\nThe `size` argument will always be provided; and will often be some large value like 0x10000. The function does not have to generate this much data, but should not generate more than that.\n\nExample function that generates a sequence of bytes, increasing from 00 to FF then repeating:\n\n```javascript\nconst a = new SimplexPair();\nvar counter = 0;\na._read = function(size){\n\tconst buf = new Uint8Array(size);\n\tfor(var i=0; i\u003csize; i++) buf[i] = (counter++)%0xFF;\n\treturn buf;\n}\n```\n\n\n## Class DuplexPair\n\n```\ninterface DuplexPair : Array {\n\t0: Duplex;\n\t1: Duplex;\n}\n```\n\nA DuplexPair is created when an application needs to return a Duplex stream; in this case, it creates a related pair of Duplex streams. What is written to the local side will become readable to the remote side; and vice-versa.\n\nMost Duplex streams will be one side of a DuplexPair. The streams are created identically, and one or the other may be used without any difference in behavior.\n\n\n## Transforming streams with DuplexPair\n\nCreating a Transform type stream with DuplexPair is trivial:\n\n```javascript\nfunction ROT13(){\n\tconst [ inside, outside ] = new DuplexPair;\n\toutside.on('readable', function(){\n\t\tfor(var buf; buf = inside.read();){\n\t\t\tinside.write(buf.toString().replace(/[a-zA-Z]/g, function(c){\n\t\t\t\tconst d = c.charCodeAt(0) + 13;\n\t\t\t\treturn String.fromCharCode( ((c\u003c=\"Z\")?90:122)\u003e=d ? d : d-26 );\n\t\t\t}));\n\t\t}\n\t});\n\treturn outside;\n}\nprocess.stdin.pipe(ROT13()).pipe(process.stdout);\n```\n\nThis forms the basis for the `Transform#initInnerSide` method below.\n\n\n## Migration\n\n0. Expose the existing `DuplexPair` class\n0. Reimplement `Duplex` as a `Source` and `Sink`\n0. Reimplement `Readable`, `Writable`, and `Transform` as ancestors of `Source` and `Sink` (see below)\n0. Implement `Source#final` property\n0. Implement `Source#close`\n0. Implement `SimplexPair`\n\n\n## Readable\n\n```\ninterface Readable : Source {\n\tWritable initWritableSide();\n\tfunction _read();\n\tfunction _destroy();\n}\n```\n\nUses `_read`, `_destroy` user-provided methods for compatibility with Streams 2.\n\nIt provides an `initWritableSide` call that allows the subclass constructor to acquire a Writable reference that fills the buffer, drained by the instance of this class.\n\n\n## Writable\n\n```\ninterface Writable : Sink {\n\tReadable initReadableSide();\n\tfunction _write(chunk, encoding, callback);\n\tfunction _writev(chunks, callback);\n\tfunction _destroy(err, callback);\n\tfunction _final(callback);\n}\n```\n\nUses `_write`, `_writev`, `_destroy`, and `_final` user-provided methods for compatibility with Streams 2.\n\nIt provides an `initReadableSide` call that allows the subclass constructor to acquire a Readable reference that drains the buffer, filled by the instance of this class.\n\n\n## Transform\n\n```\ninterface Transform : Duplex {\n\tReadable initInputSide();\n\tWritable initOutputSide();\n\tDuplex initInnerSide();\n\tfunction _transform(chunk, encoding, callback);\n\tfunction _flush(callback);\n}\n```\n\nUses `_transform` and `_flush` user-provided methods for compatibility with Streams 2.\n\nIt provides an `initInnerSide` call that allows the subclass constructor to acquire a private reference to the \"inner\" Duplex side that reads input and writes output. It may be set to a local variable, a private class property, or a public class property, as the needs of the application demand; once initialized and returned, the reference cannot be re-acquired. For example:\n\n\n```javascript\nclass ROT13 extends Transform {\n\t#inside;\n\tconstructor() {\n\t\tsuper();\n\t\tconst inside = this.#inside = this.initInnerSide();\n\t\tinside.on('readable', function(){\n\t\t\tfor(var buf; buf = inside.read();){\n\t\t\t\tinside.write(buf.toString().replace(/[a-zA-Z]/g, function(c){\n\t\t\t\t\tconst d = c.charCodeAt(0) + 13;\n\t\t\t\t\treturn String.fromCharCode( ((c\u003c=\"Z\")?90:122)\u003e=d ? d : d-26 );\n\t\t\t\t}));\n\t\t\t}\n\t\t});\n\t}\n}\nprocess.stdin.pipe(new ROT13()).pipe(process.stdout);\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fawwright%2Fnode-simplex-pair","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fawwright%2Fnode-simplex-pair","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fawwright%2Fnode-simplex-pair/lists"}