https://github.com/voxpelli/buffered-async-iterable
Buffered parallel processing of async iterables / generators
https://github.com/voxpelli/buffered-async-iterable
async-generator async-generators async-iterable async-iterables async-iterator async-iterators buffering
Last synced: 5 months ago
JSON representation
Buffered parallel processing of async iterables / generators
- Host: GitHub
- URL: https://github.com/voxpelli/buffered-async-iterable
- Owner: voxpelli
- License: mit
- Created: 2022-06-28T14:15:38.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2024-04-30T11:57:01.000Z (about 1 year ago)
- Last Synced: 2024-05-01T12:23:40.627Z (about 1 year ago)
- Topics: async-generator, async-generators, async-iterable, async-iterables, async-iterator, async-iterators, buffering
- Language: JavaScript
- Homepage:
- Size: 90.8 KB
- Stars: 6
- Watchers: 2
- Forks: 1
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Buffered parallel processing of async iterables / generators.
[](https://www.npmjs.com/package/buffered-async-iterable)
[](https://www.npmjs.com/package/buffered-async-iterable)
[](https://github.com/voxpelli/badges-cjs-esm)
[](https://github.com/voxpelli/types-in-js)
[](https://github.com/neostandard/neostandard)
[![Follow @[email protected]](https://img.shields.io/mastodon/follow/109247025527949675?domain=https%3A%2F%2Fmastodon.social&style=social)](https://mastodon.social/@voxpelli)
## Usage
### Simple
```javascript
import { bufferedAsyncMap } from 'buffered-async-iterable';
async function * asyncGenerator() {
yield ...
}
const mappedIterator = bufferedAsyncMap(asyncGenerator(), async (item) => {
// Apply additional async lookup / processing
});
for await (const item of mappedIterator) {
// Consume the buffered async iterable
}
```
### Array input
```javascript
import { bufferedAsyncMap } from 'buffered-async-iterable';
const mappedIterator = bufferedAsyncMap(['foo'], async (item) => {
// Apply additional async lookup / processing
});
for await (const item of mappedIterator) {
// Consume the buffered async iterable
}
```
### Async generator result
```javascript
import { bufferedAsyncMap } from 'buffered-async-iterable';
const mappedIterator = bufferedAsyncMap(['foo'], async function * (item) {
// Apply additional async lookup / processing
yield ...
yield * ...
});
for await (const item of mappedIterator) {
// Consume the buffered async iterable
}
```
## API
### bufferedAsyncMap()
Iterates and applies the `callback` to up to `bufferSize` items from `input` yielding values as they resolve.
#### Syntax
`bufferedAsyncMap(input, callback[, { bufferSize=6, ordered=false }]) => AsyncIterableIterator`
#### Arguments
* `input` – either an async iterable, an ordinare iterable or an array
* `callback(item)` – should be either an async generator or an ordinary async function. Items from async generators are buffered in the main buffer and the buffer is refilled by the one that has least items in the current buffer (`input` is considered equal to sub iterators in this regard when refilling the buffer)
#### Options
* `bufferSize` – _optional_ – defaults to `6`, sets the max amount of simultanoeus items that processed at once in the buffer.
* `ordered` – _optional_ – defaults to `false`, when `true` the result will be returned in order instead of unordered
### mergeIterables()
Merges all given (async) iterables in parallel, returning the values as they resolve
#### Syntax
`mergeIterables(input[, { bufferSize=6 }]) => AsyncIterableIterator`
#### Arguments
* `input` – an array of async iterables, ordinare iterables and/or arrays
#### Options
* `bufferSize` – _optional_ – defaults to `6`, sets the max amount of simultanoeus items that processed at once in the buffer.
## Similar modules
* [`hwp`](https://github.com/mcollina/hwp) – similar module by [@mcollina](https://github.com/mcollina)