https://github.com/knorpelsenf/strom
The ultimate streaming library for Deno
https://github.com/knorpelsenf/strom
async deno iterator stream
Last synced: about 1 year ago
JSON representation
The ultimate streaming library for Deno
- Host: GitHub
- URL: https://github.com/knorpelsenf/strom
- Owner: KnorpelSenf
- License: mit
- Created: 2023-06-05T21:26:51.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2025-01-09T15:22:56.000Z (over 1 year ago)
- Last Synced: 2025-03-31T02:22:09.668Z (about 1 year ago)
- Topics: async, deno, iterator, stream
- Language: TypeScript
- Homepage: https://deno.land/x/strom
- Size: 229 KB
- Stars: 6
- Watchers: 2
- Forks: 0
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# strom
The ultimate streaming library for Deno.
- completely async
- fully concurrent
- trivial to use
- built around iterators
strom lets you morph iterators in concise functional ways:
```ts
strom([3, 1, 4])
.map((x) => [x, x * x])
.filter(([, sq]) => sq < 10)
.run((pair) => console.log(pair));
// [ 3, 9 ]
// [ 1, 1 ]
```
strom is async. This means that all elements are passed lazily only once they
are needed—just like with async iterators!
In JavaScript, if you build longer chains of async iterators, this comes with a
performance penalty. Several async iterators will be run in sequence, lacking
concurrency. We will now see how strom is much faster than plain old iterators.
Let's say you have a data source that produces values slowly, such as data from
IO operations.
```ts
async function sleep() {
await new Promise((r) => setTimeout(r, 1000));
}
async function* values() {
for (const n of [3, 1, 4]) {
console.log("producing", n);
await sleep();
yield n;
}
}
```
Let's now say you want to perform more slow async ops for these values.
```ts
async function inc(n: number) {
await sleep();
return n + 1;
}
async function double(n: number) {
await sleep();
return n + n;
}
```
With iterators, it could look something like this.
```ts
async function* incItr() {
for await (const n of values()) yield await inc(n);
}
async function* doubleItr() {
for await (const n of incItr()) yield await double(n);
}
console.time("iterators");
for await (const elem of doubleItr()) {
console.log("computed", elem);
}
console.timeEnd("iterators");
```
The output is:
```bash
producing 3
computed 8
producing 1
computed 4
producing 4
computed 10
iterators: 9020ms
```
As you can see, the elements are passed through the iterators one after the
other. There is no concurrency. This is by design of the
[async iterator protocol](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_async_iterator_and_async_iterable_protocols).
Let's look at the same code written with strom:
```ts
const iter = strom(values()).map(inc).map(double);
console.time("strom");
for await (const elem of iter) {
console.log("computed", elem);
}
console.timeEnd("strom");
```
Check the output:
```bash
producing 3
computed 8
producing 1
computed 4
producing 4
computed 10
strom: 9016ms
```
So strom does the same thing as iterators by default (just in a more concise
way).
Let's speed things up by allowing strom to buffer elements. That way, it can
already fetch the next element while processing the current one, which gives us
full concurrency!
```ts
const iter = strom(values()).map(inc).map(double).parallel(5);
console.time("strom");
for await (const elem of iter) {
console.log("computed", elem);
}
console.timeEnd("strom");
```
Suddenly, it's MUCH faster:
```bash
producing 3
producing 1
producing 4
computed 8
computed 4
computed 10
strom: 5012ms
```
Whenever you are working with async iterators, you are missing out on
concurrency and readability.
Use strom.