{"id":13432489,"url":"https://github.com/pull-stream/pull-stream","last_synced_at":"2025-12-18T02:59:11.981Z","repository":{"id":7590491,"uuid":"8946748","full_name":"pull-stream/pull-stream","owner":"pull-stream","description":"minimal streams","archived":false,"fork":false,"pushed_at":"2022-12-02T10:27:04.000Z","size":273,"stargazers_count":797,"open_issues_count":24,"forks_count":63,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-08-18T08:25:57.230Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://pull-stream.github.io/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pull-stream.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-03-22T07:09:12.000Z","updated_at":"2025-08-09T06:35:09.000Z","dependencies_parsed_at":"2023-01-13T14:26:10.863Z","dependency_job_id":null,"html_url":"https://github.com/pull-stream/pull-stream","commit_stats":null,"previous_names":["dominictarr/pull-stream"],"tags_count":30,"template":false,"template_full_name":null,"purl":"pkg:github/pull-stream/pull-stream","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pull-stream%2Fpull-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pull-stream%2Fpull-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pull-stream%2Fpull-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pull-stream%2Fpull-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pull-stream","download_url":"https://codeload.github.com/pull-stream/pull-stream/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pull-stream%2Fpull-stream/sbom","scorecard":{"id":749346,"data":{"date":"2025-08-11","repo":{"name":"github.com/pull-stream/pull-stream","commit":"32972c0f5b486b4ca4242e354c39e1d71486fec7"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4.1,"checks":[{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":7,"reason":"Found 11/14 approved changesets -- score normalized to 7","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/node.js.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/node.js.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/pull-stream/pull-stream/node.js.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/node.js.yml:24: update your workflow using https://app.stepsecurity.io/secureworkflow/pull-stream/pull-stream/node.js.yml/master?enable=pin","Warn: npmCommand not pinned by hash: .github/workflows/node.js.yml:28","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   1 npmCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 28 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-22T19:47:57.082Z","repository_id":7590491,"created_at":"2025-08-22T19:47:57.082Z","updated_at":"2025-08-22T19:47:57.082Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27790069,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-18T02:00:09.725Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T02:01:12.274Z","updated_at":"2025-12-18T02:59:11.962Z","avatar_url":"https://github.com/pull-stream.png","language":"JavaScript","readme":"# pull-stream\n\nMinimal Pipeable Pull-stream\n\nIn [classic-streams](https://github.com/nodejs/node-v0.x-archive/blob/v0.8/doc/api/stream.markdown),\nstreams _push_ data to the next stream in the pipeline.\nIn [new-streams](https://github.com/nodejs/node-v0.x-archive/blob/v0.10/doc/api/stream.markdown),\ndata is pulled out of the source stream, into the destination.\n`pull-stream` is a minimal take on streams,\npull streams work great for \"object\" streams as well as streams of raw text or binary data.\n\n[![build status](https://secure.travis-ci.org/pull-stream/pull-stream.png)](https://travis-ci.org/pull-stream/pull-stream)\n\n\n## Quick Example\n\nStat some files:\n\n```js\npull(\n  pull.values(['file1', 'file2', 'file3']),\n  pull.asyncMap(fs.stat),\n  pull.collect(function (err, array) {\n    console.log(array)\n  })\n)\n```\nNote that `pull(a, b, c)` is basically the same as `a.pipe(b).pipe(c)`.\n\nTo grok how pull-streams work, read through [pull-streams workshop](https://github.com/pull-stream/pull-stream-workshop)\n\n## How do I do X with pull-streams?\n\nThere is a module for that!\n\nCheck the [pull-stream FAQ](https://github.com/pull-stream/pull-stream-faq)\nand post an issue if you have a question that is not covered.\n\n## Compatibily with node streams\n\npull-streams are not _directly_ compatible with node streams,\nbut pull-streams can be converted into node streams with\n[pull-stream-to-stream](https://github.com/pull-stream/pull-stream-to-stream)\nand node streams can be converted into pull-stream using [stream-to-pull-stream](https://github.com/pull-stream/stream-to-pull-stream)\ncorrect back pressure is preserved.\n\n### Readable \u0026 Reader vs. Readable \u0026 Writable\n\nInstead of a readable stream, and a writable stream, there is a `readable` stream,\n (aka \"Source\") and a `reader` stream (aka \"Sink\"). Through streams\nis a Sink that returns a Source.\n\nSee also:\n* [Sources](./docs/sources/index.md)\n* [Throughs](./docs/throughs/index.md)\n* [Sinks](./docs/sinks/index.md)\n\n### Source (readable stream that produces values)\n\nA Source is a function `read(end, cb)`,\nthat may be called many times,\nand will (asynchronously) call `cb(null, data)` once for each call.\n\nTo signify an end state, the stream eventually returns `cb(err)` or `cb(true)`.\nWhen signifying an end state, `data` *must* be ignored.\n\nThe `read` function *must not* be called until the previous call has called back.\nUnless, it is a call to abort the stream (`read(Error || true, cb)`).\n\n```js\nvar n = 5;\n\n// random is a source 5 of random numbers.\nfunction random (end, cb) {\n  if(end) return cb(end)\n  // only read n times, then stop.\n  if(0 \u003e --n) return cb(true)\n  cb(null, Math.random())\n}\n\n```\n\n### Sink (reader or writable stream that consumes values)\n\nA Sink is a function `reader(read)` that calls a Source (`read(null, cb)`),\nuntil it decides to stop (by calling `read(true, cb)`), or the readable ends (`read` calls\n`cb(Error || true)`\n\nAll [Throughs](./docs/throughs/index.md)\nand [Sinks](./docs/sinks/index.md)\nare reader streams.\n\n```js\n// logger reads a source and logs it.\nfunction logger (read) {\n  read(null, function next(end, data) {\n    if(end === true) return\n    if(end) throw end\n\n    console.log(data)\n    read(null, next)\n  })\n}\n```\n\nSince Sources and Sinks are functions, you can pass them to each other!\n\n```js\nlogger(random) //\"pipe\" the streams.\n\n```\n\nbut, it's easier to read if you use's pull-stream's `pull` method\n\n```js\nvar pull = require('pull-stream')\n\npull(random, logger)\n```\n\n### Creating reusable streams\n\nWhen working with pull streams it is common to create functions that return a stream.\nThis is because streams contain mutable state and so can only be used once. \nIn the above example, once `random`  has been connected to a sink and has produced 5 random numbers it will not produce any more random numbers if connected to another sink.\n\nTherefore, use a function like this to create a random number generating stream that can be reused:\n\n```js\n\n// create a stream of n random numbers\nfunction createRandomStream (n) {\n  return function randomReadable (end, cb) {\n    if(end) return cb(end)\n    if(0 \u003e --n) return cb(true)\n    cb(null, Math.random())\n  }\n}\n\npull(createRandomStream(5), logger)\n```\n\n\n### Through\n\nA through stream is both a reader (consumes values) and a readable (produces values).\nIt's a function that takes a `read` function (a Sink),\nand returns another `read` function (a Source).\n\n```js\n// double is a through stream that doubles values.\nfunction double (read) {\n  return function readable (end, cb) {\n    read(end, function (end, data) {\n      cb(end, data != null ? data * 2 : null)\n    })\n  }\n}\n\npull(createRandomStream(5), double, logger)\n```\n\n### Pipeability\n\nEvery pipeline must go from a `source` to a `sink`.\nData will not start moving until the whole thing is connected.\n\n```js\npull(source, through, sink)\n```\n\nsome times, it's simplest to describe a stream in terms of other streams.\npull can detect what sort of stream it starts with (by counting arguments)\nand if you pull together through streams, it gives you a new through stream.\n\n```js\nvar tripleThrough =\n  pull(through1(), through2(), through3())\n// The three through streams become one.\n\npull(source(), tripleThrough, sink())\n```\n\npull detects if it's missing a Source by checking function arity,\nif the function takes only one argument it's either a sink or a through.\nOtherwise it's a Source.\n\n## Duplex Streams\n\nDuplex streams, which are used to communicate between two things,\n(i.e. over a network) are a little different. In a duplex stream,\nmessages go both ways, so instead of a single function that represents the stream,\nyou need a pair of streams. `{source: sourceStream, sink: sinkStream}`\n\nPipe duplex streams like this:\n\n``` js\nvar a = duplex()\nvar b = duplex()\n\npull(a.source, b.sink)\npull(b.source, a.sink)\n\n//which is the same as\n\nb.sink(a.source); a.sink(b.source)\n\n//but the easiest way is to allow pull to handle this\n\npull(a, b, a)\n\n//\"pull from a to b and then back to a\"\n\n```\n\nThis means two duplex streams communicating actually forms two completely\nindependent pipelines:\n1. Side1's sink pulling from Side2's source\n2. Side2's sink pulling from Side1's source\n\nAs a result, one pipeline might finish or error out before or after the other.\nA duplex stream is only \"finished\" once both pipelines are done communicating.\n\n\n## Design Goals \u0026 Rationale\n\nThere is a deeper,\n[platonic abstraction](http://en.wikipedia.org/wiki/Platonic_idealism),\nwhere a streams is just an array in time, instead of in space.\nAnd all the various streaming \"abstractions\" are just crude implementations\nof this abstract idea.\n\n[classic-streams](https://github.com/joyent/node/blob/v0.8.16/doc/api/stream.markdown),\n[new-streams](https://github.com/joyent/node/blob/v0.10/doc/api/stream.markdown),\n[reducers](https://github.com/Gozala/reducers)\n\nThe objective here is to find a simple realization of the best features of the above.\n\n### Type Agnostic\n\nA stream abstraction should be able to handle both streams of text and streams\nof objects.\n\n### A pipeline is also a stream.\n\nSomething like this should work: `a.pipe(x.pipe(y).pipe(z)).pipe(b)`\nthis makes it possible to write a custom stream simply by\ncombining a few available streams.\n\n### Propagate End/Error conditions.\n\nIf a stream ends in an unexpected way (error),\nthen other streams in the pipeline should be notified.\n(this is a problem in node streams - when an error occurs,\nthe stream is disconnected, and the user must handle that specially)\n\nAlso, the stream should be able to be ended from either end.\n\n### Transparent Backpressure \u0026 Laziness\n\nVery simple transform streams must be able to transfer back pressure\ninstantly.\n\nThis is a problem in node streams, pause is only transfered on write, so\non a long chain (`a.pipe(b).pipe(c)`), if `c` pauses, `b` will have to write to it\nto pause, and then `a` will have to write to `b` to pause.\nIf `b` only transforms `a`'s output, then `a` will have to write to `b` twice to\nfind out that `c` is paused.\n\n[reducers](https://github.com/Gozala/reducers) reducers has an interesting method,\nwhere synchronous tranformations propagate back pressure instantly!\n\nThis means you can have two \"smart\" streams doing io at the ends, and lots of dumb\nstreams in the middle, and back pressure will work perfectly, as if the dumb streams\nare not there.\n\nThis makes laziness work right.\n\n### handling end, error, and abort.\n\nin pull streams, any part of the stream (source, sink, or through)\nmay terminate the stream. (this is the case with node streams too,\nbut it's not handled well).\n\n#### source: end, error\n\nA source may end (`cb(true)` after read) or error (`cb(error)` after read)\nAfter ending, the source *must* never `cb(null, data)`\n\n#### sink: abort\n\nSinks do not normally end the stream, but if they decide they do\nnot need any more data they may \"abort\" the source by calling `read(true, cb)`.\nA abort (`read(true, cb)`) may be called before a preceding read call\nhas called back.\n\n### handling end/abort/error in through streams\n\nRules for implementing `read` in a through stream:\n1) Sink wants to stop. sink aborts the through\n\n    just forward the exact read() call to your source,\n    any future read calls should cb(true).\n\n2) We want to stop. (abort from the middle of the stream)\n\n    abort your source, and then cb(true) to tell the sink we have ended.\n    If the source errored during abort, end the sink by cb read with `cb(err)`.\n    (this will be an ordinary end/error for the sink)\n\n3) Source wants to stop. (`read(null, cb) -\u003e cb(err||true)`)\n\n    forward that exact callback towards the sink chain,\n    we must respond to any future read calls with `cb(err||true)`.\n\nIn none of the above cases data is flowing!\n4) If data is flowing (normal operation:   `read(null, cb) -\u003e cb(null, data)`\n\n    forward data downstream (towards the Sink)\n    do none of the above!\n\nThere either is data flowing (4) OR you have the error/abort cases (1-3), never both.\n\n\n## 1:1 read-callback ratio\n\nA pull stream source (and thus transform) returns *exactly one value* per read.\n\nThis differs from node streams, which can use `this.push(value)` and in internal\nbuffer to create transforms that write many values from a single read value.\n\nPull streams don't come with their own buffering mechanism, but [there are ways\nto get around this](https://github.com/dominictarr/pull-stream-examples/blob/master/buffering.js).\n\n\n## Minimal bundle\n\nIf you need only the `pull` function from this package you can reduce the size\nof the imported code (for instance to reduce a Browserify bundle) by requiring\nit directly:\n\n\n```js\nvar pull = require('pull-stream/pull')\n\npull(createRandomStream(5), logger())\n```\n\n\n## Further Examples\n\n- [dominictarr/pull-stream-examples](https://github.com/dominictarr/pull-stream-examples)\n- [./docs/examples](./docs/examples.md)\n\nExplore this repo further for more information about\n[sources](./docs/sources/index.md),\n[throughs](./docs/throughs/index.md),\n[sinks](./docs/sinks/index.md), and\n[glossary](./docs/glossary.md).\n\n\n## License\n\nMIT\n\n\n","funding_links":[],"categories":["JavaScript","Legacy"],"sub_categories":["[`pull-level`][pull-level]"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpull-stream%2Fpull-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpull-stream%2Fpull-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpull-stream%2Fpull-stream/lists"}