{"id":21747468,"url":"https://github.com/charlespascoe/object-streaming","last_synced_at":"2026-05-19T10:39:02.477Z","repository":{"id":57312774,"uuid":"102644054","full_name":"charlespascoe/object-streaming","owner":"charlespascoe","description":"Type-safe Object Streaming in JavaScript/TypeScript","archived":false,"fork":false,"pushed_at":"2017-09-13T19:32:29.000Z","size":97,"stargazers_count":0,"open_issues_count":3,"forks_count":0,"subscribers_count":1,"default_branch":"develop","last_synced_at":"2025-07-12T05:45:03.709Z","etag":null,"topics":["typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/charlespascoe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-09-06T18:30:05.000Z","updated_at":"2017-09-07T08:21:28.000Z","dependencies_parsed_at":"2022-09-20T23:02:37.944Z","dependency_job_id":null,"html_url":"https://github.com/charlespascoe/object-streaming","commit_stats":null,"previous_names":["cpascoe95/object-streaming"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/charlespascoe/object-streaming","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charlespascoe%2Fobject-streaming","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charlespascoe%2Fobject-streaming/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charlespascoe%2Fobject-streaming/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charlespascoe%2Fobject-streaming/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/charlespascoe","download_url":"https://codeload.github.com/charlespascoe/object-streaming/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charlespascoe%2Fobject-streaming/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265124249,"owners_count":23714899,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["typescript"],"created_at":"2024-11-26T08:09:14.322Z","updated_at":"2026-05-19T10:39:02.470Z","avatar_url":"https://github.com/charlespascoe.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![npm](https://img.shields.io/npm/v/object-streaming.svg)](https://www.npmjs.com/package/object-streaming)\n[![npm](https://img.shields.io/npm/dt/object-streaming.svg)](https://www.npmjs.com/package/object-streaming)\n[![npm](https://img.shields.io/npm/l/object-streaming.svg)](https://www.npmjs.com/package/object-streaming)\n\n# Type-safe Object Streaming\n\nA type-safe framework for stream processing of objects in TypeScript, useful for efficiently processing streams of data such as reading from a database using a cursor.\n\nThe framework provides a number of useful built-in streams, and you can easily create your own.\n\n- [Installation](#installation)\n- [Usage](#usage)\n- [Concepts](#concepts)\n- [Built-In Utility Streams](#built-in-utility-streams)\n    - [forEach](#foreachcallback)\n    - [map](#mapcallback)\n    - [filter](#filtercallback)\n    - [branch](#branchcallback-altstream)\n    - [split](#splitstreams)\n    - [merge](#mergestreams)\n    - [batch](#batchoptions)\n    - [spread](#spread)\n- [Custom Streams](#custom-streams)\n\n# Installation\n\n`$ npm install --save object-streaming`\n\n# Usage\n\nThese examples are written in TypeScript, but this module is also completely compatible with plain JavaScript - essentially just remove any type definitions.\n\nHere's a stream that processes numbers:\n\n```typescript\nlet strm = source\u003cnumber\u003e();\n\nstrm\n  .pipe(filter((x: number) =\u003e x % 7 !== 0))\n  .pipe(map((x: number) =\u003e x * 2))\n  .pipe(batch({maxItems: 3, idleTimeout: 100}))\n  .pipe(forEach((array: number[]) =\u003e {\n    console.log('Batched array length:', array.length);\n  }))\n  .pipe(spread())\n  .pipe(branch(\n    (x: number) =\u003e x % 10 === 0,\n    forEach((x: number) =\u003e console.log(`[${x}]`))\n  ))\n  .pipe(forEach((x: number) =\u003e console.log(x)));\n\n\n\nfor (let i = 0; i \u003c 20; i++) {\n  strm.input(i);\n}\n\n// Output:\n// Batched array length: 3\n// 2\n// 4\n// 6\n// Batched array length: 3\n// 8\n// [10]\n// 12\n// Batched array length: 3\n// 16\n// 18\n// [20]\n// Batched array length: 3\n// 22\n// 24\n// 26\n// Batched array length: 3\n// [30]\n// 32\n// 34\n// Batched array length: 2\n// 36\n// 38\n```\n\n# Concepts\n\nComplex processing streams can be constructed by combining multiple simple stream objects. Stream objects define:\n\n- An input type\n- An `input` method\n- An output type\n- An `output` method\n- A `pipe` method\n\nA stream accepts an object when `input` is called, does some processing on it, and calls `output` with the results of the processing. A stream can output objects to zero or more streams. `strmA.pipe(strmB)` will cause all outputted objects of `strmA` to be inputted into `strmB`; `pipe` will then return `strmB` so that `strmB`'s output can be piped into something else. This allows for a clean method-chaining API.\n\nSince complex streams are made up of multiple stream objects, the entry and exit stream objects will be different. Therefore, in the following code, `strm` will be a reference to the *last* stream object, not the first:\n\n```typescript\nlet strm = source\u003cnumber\u003e()\n   .pipe(/* some other stream */)\n   .pipe(forEach((x: number) =\u003e console.log(x))); // \u003c- 'strm' has a reference to the stream created by this forEach(), not source()\n```\n\nIf you need both the entry and exit streams, define the entry stream first, then compose the composite stream:\n\n```typescript\nlet entryStrm = source\u003cnumber\u003e();\n\nlet exitStrm = entryStrm\n  .pipe(/* ... */)\n  .pipe(/* ... */)\n  .pipe(/* ... */)\n  .pipe(/* ... */);\n\n// Later in the code...\n\nexitStrm.pipe(forEach(x =\u003e console.log(x)));\nentryStrm.input('xyz');\n```\n\nMore conveniently, you can use the `stream` utility function to return a single Stream object, which takes a function with a single argument of the entry (source) stream.\n\n```typescript\nlet strm = stream\u003cnumber,string\u003e(src =\u003e src\n  .pipe(/* ... */)\n  .pipe(/* ... */)\n  .pipe(/* ... */)\n  .pipe(/* ... */)\n);\n\nstrm.pipe(forEach(x =\u003e console.log(x)));\nstrm.input('xyz');\n```\n\n# Built-In Utility Streams\n\n## forEach(*callback*)\n\nFor each item that passes through the stream, call the callback, and then pass the item on to the next stream:\n\n```typescript\nimport { forEach } from 'object-stream';\n\n// ...\n\nstrm\n  .pipe(forEach((item: SomeClass) =\u003e {\n    item.foo = true;\n    item.bar();\n  });\n```\n\n## map(*callback*)\n\nMap each item that passes through the stream using the callback, and pass it onto the next stream:\n\n```typescript\nimport { map } from 'object-stream';\n\n// ...\n\nstrm\n  .pipe(map((num: number) =\u003e new String(num)))\n```\n\n## filter(*callback*)\n\nPass each item into the callback; if it returns true, pass it onto the next stream, otherwise drop the item:\n\n```typescript\nimport { filter } from 'object-stream';\n\n// ...\n\nstrm\n   .pipe(filter((item: SomeClass) =\u003e item.isFlagChecked()))\n```\n\n## branch(*callback*, *altStream*)\n\nPass each item into the callback; if it returns true, pass it onto the alternate stream, otherwise pass it onto the next stream:\n\n```typescript\nimport { branch } from 'object-stream';\n\n// ...\n\nstrm\n  .pipe(branch(\n    (item: SomeClass) =\u003e true,\n    forEach((item: SomeClass) =\u003e console.log(item.foo))\n  ))\n  .pipe(forEach((item: SomeClass) =\u003e {\n    // This will never run because the branch callback always returns true!\n  }));\n```\n\n## split(*...streams*)\n\nEach input item is passed to all the given streams, and then passed to the next stream:\n\n```typescript\nimport { split } from 'object-stream';\n\n// ...\n\nstrm\n  .pipe(split(\n    forEach((num: number) =\u003e console.log('A', num))\n    forEach((num: number) =\u003e console.log('B', num))\n    forEach((num: number) =\u003e console.log('C', num))\n  ))\n  .pipe(forEach((num: number) =\u003e console.log(num)))\n\nstrm.input(123)\n\n// Outputs:\n// A 123\n// B 123\n// C 123\n// 123\n```\n\n## merge(*...streams*)\n\nEach item outputted by the given streams is passed to the next stream:\n\n```typescript\nimport { merge } from 'object-stream';\n\n// ...\n\nlet strmA = source\u003cnumber\u003e(),\n    strmB = source\u003cnumber\u003e();\n\nmerge(strmA, strmB)\n  .pipe(forEach((num: number) =\u003e console.log(num)))\n\nstrmA.input(123);\nstrmB.input(456);\n\n// Output:\n// 123\n// 456\n```\n\n`merge` can also be used in-line:\n\n```typescript\nlet strmA = source\u003cnumber\u003e(),\n    strmB = source\u003cnumber\u003e(),\n    strmC = source\u003cnumber\u003e();\n\nstrmA\n  .pipe(map((num: number) =\u003e num * 2))\n  .pipe(merge(strmA, strmB))\n  .pipe(forEach((num: number) =\u003e console.log(num)));\n\nstrmA.input(123);\nstrmA.input(456);\nstrmA.input(789);\n\n// Output:\n// 246\n// 456\n// 789\n```\n\n## batch(*options*)\n\nGroups together multiple items into an array:\n\n```typescript\nimport { batch } from 'object-stream';\n\n// ...\n\nstrm\n  .pipe(batch({maxItems: 4, idleTimeout: 100}))\n  .pipe(forEach((nums: number[]) =\u003e console.log(nums)))\n\nfor (let i = 0; i \u003c 10; i++) {\n  strm.input(i);\n}\n\n// Output:\n// [0, 1, 2, 3]\n// [4, 5, 6, 7]\n// [8, 9]\n```\n\nThe options affect the behaviour of the batcher:\n\n- `maxItems: number` - when the number of items in the batch reaches `maxItems`, then emit the batch\n- `idleTimeout: number`  - after receiving an item, when no new items have been received for the duration of the idle timeout (in milliseconds), emit the batch\n- `delayTimeout: number` - after receiving the first item into an empty batch, set a timer for the specified duration (in milliseconds); when the timer fires, emit the batch with all the items collected in that time\n\n`idleTimeout` and `delayTimeout` can't be used together.\n\nWhen no options are provided, it defaults to `idleTimeout = 0`, which emits the batch on the next event loop iteration.\n\n## spread()\n\nTakes an array an passes each item onto the next stream:\n\n```typescript\nimport { spread } from 'object-stream';\n\n// ...\n\nlet strm = source\u003cnumber[]\u003e();\n\nstrm\n  .pipe(spread())\n  .pipe(forEach((num: number) =\u003e console.log(num)));\n```\n\n# Custom Streams\n\nThere are two main classes for defining custom streams: `SourceStream\u003cT\u003e` and `Stream\u003cI,O\u003e`.\n\n`SourceStream` is used when a class only outputs objects. Call `this.output` to emit an object:\n\n```typescript\nimport { SourceStream } from 'object-stream';\n\n// ...\n\nclass BasicClockStream extends SourceStream\u003cnumber\u003e {\n  constructor() {\n    super();\n\n    setInterval(() =\u003e this.output(Date.now()), 1000);\n  }\n}\n\nlet clockStrm = new BasicClockStream();\n\nclockStrm.pipe(/* another stream that accepts 'number' */);\n```\n\n`Stream` is used when to build stream processing classes that take an input and emit an output. It must define a public `input` method which accepts a single argument of the specified input type:\n\n```typescript\nimport { Stream } from 'object-stream';\n\n// ...\n\nclass IntParseStream extends Stream\u003cstring,number\u003e {\n  public input(obj: string) {\n    if (/\\d+/.test(obj)) {\n      this.output(parseInt(obj));\n    }\n  }\n}\n\nlet ips = new IntParseStream();\n\nips\n  .pipe(forEach((num: number) =\u003e console.log(num)));\n\nips.input('123');\nips.input('abc');\nips.input('456');\n\n// Output:\n// 123\n// 456\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharlespascoe%2Fobject-streaming","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcharlespascoe%2Fobject-streaming","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharlespascoe%2Fobject-streaming/lists"}