{"id":17482432,"url":"https://github.com/creationix/min-stream","last_synced_at":"2025-07-20T14:32:04.941Z","repository":{"id":8612652,"uuid":"10253356","full_name":"creationix/min-stream","owner":"creationix","description":"A meta-package for min-stream helper modules.","archived":false,"fork":false,"pushed_at":"2013-05-24T16:12:05.000Z","size":188,"stargazers_count":33,"open_issues_count":1,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-24T04:14:50.016Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/creationix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-05-23T21:00:27.000Z","updated_at":"2023-09-08T16:39:37.000Z","dependencies_parsed_at":"2022-08-27T17:02:19.761Z","dependency_job_id":null,"html_url":"https://github.com/creationix/min-stream","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Fmin-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Fmin-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Fmin-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/creationix%2Fmin-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/creationix","download_url":"https://codeload.github.com/creationix/min-stream/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247684454,"owners_count":20979071,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-18T23:43:12.096Z","updated_at":"2025-04-10T02:44:18.760Z","avatar_url":"https://github.com/creationix.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"min-stream\n==========\n\n[![node support](https://travis-ci.org/creationix/min-stream.png)](https://travis-ci.org/creationix/min-stream)\n\n[![browser support](https://ci.testling.com/creationix/min-stream.png)](https://ci.testling.com/creationix/min-stream)\n\n\nA meta-package for min-stream helper modules.  This contains several useful and related modules for working with min-streams.  Note that implementations of min-stream should not need to depend on this library.\n\n## The Interface\n\nThe min-stream system is an interface more than anything.  This has three main types: sources, filters, and sinks.\n\n### Source\n\nA source is a place where data comes from.  Since this is a pull-stream system, it's the place we pull data from.  It has the JavaScript signature:\n\n```js\nfunction read(close, callback) {\n  // If close is truthy, that means to clean up any resources and close the stream\n  // call callback with an END event when done.\n  // Otherwise, get some data and when ready, callback(err, item)\n  // DATA  is encoded as (falsy, item)\n  // END   is encoded as (falsy, undefined) or (falsy) or ()\n  // ERROR is encoded as (err, undefined)   or (err)\n}\n```\n\nSources are usually things like the readable end of TCP sockets and readable file streams.  They can really be anything that emits events in a stream though.\n\n### Filter\n\nA filter is a function that accepts a source and returns a new transformed source.  This is where protocols are implemented.  It has the signature:\n\n```js\nfunction filter(read) {\n  // Set up per-stream state here\n  return function (close, callback) {\n    // Handle per event logic here, reading from upstream `read` when needed.\n    // Close is often just forwarded to the upstream `read`.\n  };\n}\n```\n\nThere are also technically two other filter types supported by the `chain` helper described later on.  They are regular map functions and push filters.  They have less power than normal pull filters; but are in many cases much easier to write.\n\n### Sink\n\nA sink represents something like the writable end of a TCP socket or a writable file.  You can't write directly to it.  Rather, you hand it a source function and it pulls at the rate it can handle.  This way backpressure works automatically without having to deal with pause, resume, drain, and ready.\n\n```js\n// Usage is simple.\nsink(source);\n```\n\nOr in the likely case you have a filter\n\n```js\nsink(\n  filter(\n    source\n  )\n);\n```\n\n## Modules in this Package\n\nNow that you know what min-streams are, you'll find that working with them directly is sometimes challenging.  Their goal was to be minimal and easy to implement.  This library makes them easy to use and very powerful as well!\n\nAny module can be loaded either directly as `require('min-stream/module.js')` or as a reference to index as `require('min-stream').module` where `module` is (`chain`, `cat`, `merge`, `dup`, `mux`, `demux`, `consume`, or `array`);\n\n### Chain\n\nManually connecting sink to filter to source ends up being a reverse pyramid.  Often it's preferred to write it as a chain from source to sink in the same direction the data flows.\n\nThe `chain` module helps with this.  Here is a simple example.\n\n```js\nchain\n  .source(socket.source)\n  .pull(myapp)\n  .sink(socket.sink);\n\nfunction myapp(read) {\n  return function (close, callback) {\n    // Implement app logic as pull filter...\n  };\n}\n```\n\n#### chain.source(source) -\u003e source\n\nWrap a source function adding in the `pull`, `push`, `map`, and `sink` methods.\n\nReturns the function for easy chaining.\n\n#### chain.pull(pull) -\u003e pull\n\nWrap a pull filter by adding in the `pull`, `push`, `map`, and `sink` methods.\n\nReturns the function for easy chaining.\n\n#### chain.push(push) -\u003e pull\n\nWrap a push filter, converting it to a pull filter.  Push filters accept an emit function and return a new emit function.  There is no way to control back-pressure from within a push filter.  Also close events skip push filters.\n\n```js\nfunction (emit) {\n  // Set up per-stream state\n  return function (err, item) {\n    // handle this event and call `emit` 0 or more times as required by protocol.\n  };\n}\n```\n\n#### chain.map(map) -\u003e pull\n\nWrap a map function, converting it to a pull filter.  Map functions can't see backpressure, close events, or even end or error events.  They only see items.  Map functions are stateless.  If they throw an exception it will be caught and sent as an error event.\n\n```js\nfunction (item) {\n  // return transformed item.\n}\n```\n\nExisting examples of map functions are `JSON.stringify` and `JSON.parse`.\n\n\n#### source.pull(pull) -\u003e source, source.push(push) -\u003e source, source.map(map) -\u003e source\n\nWhen chaining off a wrapped source, you can add pull, push, or map filters and a new wrapped source will be returned every time.\n\n```js\nvar parsedSource = chain\n  .source(socket.source)\n  .push(deframer)\n  .map(JSON.parse);\n```\n\n#### source.sink(sink)\n\nAttaching a sink to a source completes the chain and starts the action.  Doesn't return anything and can't be chained from.\n\n```js\nchain\n  .source(file.source)\n  .map(capitalize)\n  .sink(file.sink);\n```\n\n#### pull.pull(pull) -\u003e pull, pull.push(push) -\u003e pull, pull.map(map) -\u003e pull\n\nWhen chaining off a wrapped pull filter, you can add pull, push, or map filters and a new composite and wrapped pull filter will be returned.\n\n```js\nvar combined = chain\n  .push(pushFilter)\n  .map(myMap);\n```\n\n#### pull.sink(sink) -\u003e sink\n\nA composite sink can be built by chaining a sink call from a wrapped pull filter.\n\n```js\nvar newsink = chain\n  .map(JSON.stringify)\n  .sink(socket.sink);\n```\n\n### cat(source1, source2, ..) -\u003e source\n\nAccepts a variable number of source functions and returns a combined source.\nThe input sources will be read in sequential order.\nArrays of items may be used in place of sources for convenience.\n\n```js\nvar combined = cat([\"header\"], stream, [\"tail\"]);\n```\n\n### merge(source1, source2, ...) -\u003e source\n\nAccepts a variable number of source functions and returns a combined source.\nThe input sources will be read in parallel.\n\n### dup(num, source) -\u003e sources\n\nDuplicates input `source` into `num` copies. Returns the copies as an array of sources.  This does not buffer, so all the duplicates must be read in parallel.\n\n```js\n// Split a stream to go to both file and socket\nvar sources = dup(2, input);\nlogfile.sink(sources[0]);\ntcpClient.sink(sources[1]);\n```\n\n### mux(streams) -\u003e stream\n\nMultiplex several streams into one stream.  This is like merge, except it annotates the data events so that you know which source they came from.\n\nIt accepts either an array of streams or a hash of streams.  In the case of an array, the items will be tagged with the numerical index of the stream.  In case of a hash object, they will be tagged with the object keys.\n\nItems will be tagged by wrapping them in an array.  For example, the item `\"Hello\"` from the stream `words` would be `[\"words\", \"Hello\"]` in the combined stream.\n\n```js\n// tag using 0 and 1\nvar combined = mux([input, output]);\n// or use words\nvar combined = mux({\n  input: input,\n  output: output\n});\n```\n\n### demux(names, stream) -\u003e streams\n\nDe-multiplex a stream.  This module undoes what the `mux` module did.  You have to give it either the number of array streams to expect or the hash keys as an array.  Any keys you leave out will be dropped in the stream.\n\n```js\nvar sources = demux([\"words\", \"colors\"], combined);\nsources.words // -\u003e a stream of anything with the pattern [\"words\", value]\nsources.colors // -\u003e another stream of data\n```\n\nJust like dup, all the output streams must be read in parallel.  There is no internal buffering.\n\nHere is an example of array mode:\n\n```js\nvar sources = demux(2, combined);\nsources[0] // items matching [0, item] as just item\nsources[1] // items matching [1, item] as just item\n```\n\n### consume(source, callback)\n\nThis helper will consume all the events in a stream and `callback(null, items)` with an array of items when done.  If there was an error, it will be in the `callback(err)`.\n\n#### consume.sink(callback) -\u003e sink\n\nConsume is also available as a sink for use with `chain`.\n\n#### consume.all(sources, callback)\n\nThis is like `consume`, except it accepts an array of streams of a hash of streams.  It will consume them all in parallel and `callback(null, result)` when all are done.  The `result` will have the same structure and type of the original `sources`.\n\n```js\nconsume.all(demux(5, source), function (err, result) {\n  // result is an array of arrays\n});\n```\n\n### array(array) -\u003e source\n\nA simple module that converts an array of items into a source function.\n\n#### array.async(array) -\u003e source\n\nA version of the conversion that produces a slow source that waits a random amount of time before calling the read callback each time.  Useful in testing.\n\n## Related packages\n\nThere are many packaged and modules out there that implement this interface.  Some interesting ones are:\n\n - [min-stream-node][] - A node.js adapter that provides tcp client and server as well as file streams using min-streams.\n - [min-stream-uv][] - A crazy experiment to implement the same interface as min-stream-node, but using node's private internal libuv bindings for maximum speed and unstability.\n - [min-stream-chrome][] - Another implementation of the tcp and fs API, but wrapping chrome packaged apps's special APIs.\n - [min-stream-http-codec][] - A set of filters that makes implementing HTTP server and client programs easy on top of the tcp adapters.\n - [js-git][] - The project that started all this.  An implementation of git in JavaScript.  Uses min-streams throughout.\n\n[min-stream-node]: https://github.com/creationix/min-stream-node\n[min-stream-uv]: https://github.com/creationix/min-stream-uv\n[min-stream-chrome]: https://github.com/creationix/min-stream-chrome\n[min-stream-http-codec]: https://github.com/creationix/min-stream-http-codec\n[js-git]: https://github.com/creationix/js-git\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcreationix%2Fmin-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcreationix%2Fmin-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcreationix%2Fmin-stream/lists"}