Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/sindresorhus/get-stream
Get a stream as a string, Buffer, ArrayBuffer or array
https://github.com/sindresorhus/get-stream
Last synced: 4 days ago
JSON representation
Get a stream as a string, Buffer, ArrayBuffer or array
- Host: GitHub
- URL: https://github.com/sindresorhus/get-stream
- Owner: sindresorhus
- License: mit
- Created: 2015-10-14T17:22:22.000Z (over 9 years ago)
- Default Branch: main
- Last Pushed: 2024-10-28T05:15:33.000Z (3 months ago)
- Last Synced: 2024-10-29T15:46:34.474Z (3 months ago)
- Language: JavaScript
- Homepage:
- Size: 151 KB
- Stars: 341
- Watchers: 11
- Forks: 33
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
- License: license
- Security: .github/security.md
Awesome Lists containing this project
- awesome-nodejs-cn - get-stream - 以字符串或 Buffer 的形式获取流 (包 / 文件流)
- awesome-nodejs - get-stream - Get a stream as a string, buffer, or array.  (Repository / Streams)
- awesome-nodejs - get-stream - Get a stream as a string or buffer. (Packages / Streams)
- awesome-nodejs - get-stream - Get a stream as a string, buffer, or array - ★ 131 (Streams)
- awesome-nodejs-streams - sindresorhus/get-stream - Get a stream as a string, buffer, or array (Modules)
- awesome-node - get-stream - Get a stream as a string or buffer. (Packages / Streams)
- awesome-nodejs-cn - get-stream - 获取流作为字符串或缓冲区. (目录 / 流处理)
- awesome-ava - get-stream - Test the output of streams. [(Example)](https://github.com/sindresorhus/ora/blob/4ceeedd51795bb88a8033229d198e70cd8a2aff7/test.js#L33-L35) (Works with AVA)
- awesome-nodejs-cn - get-stream - **star:344** 获取一个流作为字符串或缓冲区 (包 / 流)
README
# get-stream
> Get a stream as a string, Buffer, ArrayBuffer or array
## Features
- Works in any JavaScript environment ([Node.js](#nodejs-streams), [browsers](#browser-support), etc.).
- Supports [text streams](#getstreamstream-options), [binary streams](#getstreamasbufferstream-options) and [object streams](#getstreamasarraystream-options).
- Supports [async iterables](#async-iterables).
- Can set a [maximum stream size](#maxbuffer).
- Returns [partially read data](#errors) when the stream errors.
- [Fast](#benchmarks).
## Install
```sh
npm install get-stream
```
## Usage
### Node.js streams
```js
import fs from 'node:fs';
import getStream from 'get-stream';
const stream = fs.createReadStream('unicorn.txt');
console.log(await getStream(stream));
/*
,,))))))));,
__)))))))))))))),
\|/ -\(((((''''((((((((.
-*-==//////(('' . `)))))),
/|\ ))| o ;-. '((((( ,(,
( `| / ) ;))))' ,_))^;(~
| | | ,))((((_ _____------~~~-. %,;(;(>';'~
o_); ; )))(((` ~---~ `:: \ %%~~)(v;(`('~
; ''''```` `: `:::|\,__,%% );`'; ~
| _ ) / `:|`----' `-'
______/\/~ | / /
/~;;.____/;;' / ___--,-( `;;;/
/ // _;______;'------~~~~~ /;;/\ /
// | | / ; \;;,\
(<_ | ; /',/-----' _>
\_| ||_ //~;~~~~~~~~~
`\_| (,~~
\~\
~~
*/
```
### Web streams
```js
import getStream from 'get-stream';
const {body: readableStream} = await fetch('https://example.com');
console.log(await getStream(readableStream));
```
This works in any browser, even [the ones](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream#browser_compatibility) not supporting `ReadableStream.values()` yet.
### Async iterables
```js
import {opendir} from 'node:fs/promises';
import {getStreamAsArray} from 'get-stream';
const asyncIterable = await opendir(directory);
console.log(await getStreamAsArray(asyncIterable));
```
## API
The following methods read the stream's contents and return it as a promise.
### getStream(stream, options?)
`stream`: [`stream.Readable`](https://nodejs.org/api/stream.html#class-streamreadable), [`ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream), or [`AsyncIterable`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_async_iterator_and_async_iterable_protocols)\
`options`: [`Options`](#options)
Get the given `stream` as a string.
### getStreamAsBuffer(stream, options?)
Get the given `stream` as a Node.js [`Buffer`](https://nodejs.org/api/buffer.html#class-buffer).
```js
import {getStreamAsBuffer} from 'get-stream';
const stream = fs.createReadStream('unicorn.png');
console.log(await getStreamAsBuffer(stream));
```
### getStreamAsArrayBuffer(stream, options?)
Get the given `stream` as an [`ArrayBuffer`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer).
```js
import {getStreamAsArrayBuffer} from 'get-stream';
const {body: readableStream} = await fetch('https://example.com');
console.log(await getStreamAsArrayBuffer(readableStream));
```
### getStreamAsArray(stream, options?)
Get the given `stream` as an array. Unlike [other methods](#api), this supports [streams of objects](https://nodejs.org/api/stream.html#object-mode).
```js
import {getStreamAsArray} from 'get-stream';
const {body: readableStream} = await fetch('https://example.com');
console.log(await getStreamAsArray(readableStream));
```
#### options
Type: `object`
##### maxBuffer
Type: `number`\
Default: `Infinity`
Maximum length of the stream. If exceeded, the promise will be rejected with a `MaxBufferError`.
Depending on the [method](#api), the length is measured with [`string.length`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/length), [`buffer.length`](https://nodejs.org/api/buffer.html#buflength), [`arrayBuffer.byteLength`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer/byteLength) or [`array.length`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/length).
## Errors
If the stream errors, the returned promise will be rejected with the `error`. Any contents already read from the stream will be set to `error.bufferedData`, which is a `string`, a `Buffer`, an `ArrayBuffer` or an array depending on the [method used](#api).
```js
import getStream from 'get-stream';
try {
await getStream(streamThatErrorsAtTheEnd('unicorn'));
} catch (error) {
console.log(error.bufferedData);
//=> 'unicorn'
}
```
## Browser support
For this module to work in browsers, a bundler must be used that either:
- Supports the [`exports.browser`](https://nodejs.org/api/packages.html#community-conditions-definitions) field in `package.json`
- Strips or ignores `node:*` imports
Most bundlers (such as [Webpack](https://webpack.js.org/guides/package-exports/#target-environment)) support either of these.
Additionally, browsers support [web streams](#web-streams) and [async iterables](#async-iterables), but not [Node.js streams](#nodejs-streams).
## Tips
### Alternatives
If you do not need the [`maxBuffer`](#maxbuffer) option, [`error.bufferedData`](#errors), nor browser support, you can use the following methods instead of this package.
#### [`streamConsumers.text()`](https://nodejs.org/api/webstreams.html#streamconsumerstextstream)
```js
import fs from 'node:fs';
import {text} from 'node:stream/consumers';
const stream = fs.createReadStream('unicorn.txt', {encoding: 'utf8'});
console.log(await text(stream))
```
#### [`streamConsumers.buffer()`](https://nodejs.org/api/webstreams.html#streamconsumersbufferstream)
```js
import {buffer} from 'node:stream/consumers';
console.log(await buffer(stream))
```
#### [`streamConsumers.arrayBuffer()`](https://nodejs.org/api/webstreams.html#streamconsumersarraybufferstream)
```js
import {arrayBuffer} from 'node:stream/consumers';
console.log(await arrayBuffer(stream))
```
#### [`readable.toArray()`](https://nodejs.org/api/stream.html#readabletoarrayoptions)
```js
console.log(await stream.toArray())
```
#### [`Array.fromAsync()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/fromAsync)
If your [environment supports it](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/fromAsync#browser_compatibility):
```js
console.log(await Array.fromAsync(stream))
```
### Non-UTF-8 encoding
When all of the following conditions apply:
- [`getStream()`](#getstreamstream-options) is used (as opposed to [`getStreamAsBuffer()`](#getstreamasbufferstream-options) or [`getStreamAsArrayBuffer()`](#getstreamasarraybufferstream-options))
- The stream is binary (not text)
- The stream's encoding is not UTF-8 (for example, it is UTF-16, hexadecimal, or Base64)
Then the stream must be decoded using a transform stream like [`TextDecoderStream`](https://developer.mozilla.org/en-US/docs/Web/API/TextDecoderStream) or [`b64`](https://github.com/hapijs/b64).
```js
import getStream from 'get-stream';
const textDecoderStream = new TextDecoderStream('utf-16le');
const {body: readableStream} = await fetch('https://example.com');
console.log(await getStream(readableStream.pipeThrough(textDecoderStream)));
```
### Blobs
[`getStreamAsArrayBuffer()`](#getstreamasarraybufferstream-options) can be used to create [Blobs](https://developer.mozilla.org/en-US/docs/Web/API/Blob).
```js
import {getStreamAsArrayBuffer} from 'get-stream';
const stream = fs.createReadStream('unicorn.txt');
console.log(new Blob([await getStreamAsArrayBuffer(stream)]));
```
### JSON streaming
[`getStreamAsArray()`](#getstreamasarraystream-options) can be combined with JSON streaming utilities to parse JSON incrementally.
```js
import fs from 'node:fs';
import {compose as composeStreams} from 'node:stream';
import {getStreamAsArray} from 'get-stream';
import streamJson from 'stream-json';
import streamJsonArray from 'stream-json/streamers/StreamArray.js';
const stream = fs.createReadStream('big-array-of-objects.json');
console.log(await getStreamAsArray(
composeStreams(stream, streamJson.parser(), streamJsonArray.streamArray()),
));
```
## Benchmarks
### Node.js stream (100 MB, binary)
- `getStream()`: 142ms
- `text()`: 139ms
- `getStreamAsBuffer()`: 106ms
- `buffer()`: 83ms
- `getStreamAsArrayBuffer()`: 105ms
- `arrayBuffer()`: 81ms
- `getStreamAsArray()`: 24ms
- `stream.toArray()`: 21ms
### Node.js stream (100 MB, text)
- `getStream()`: 90ms
- `text()`: 89ms
- `getStreamAsBuffer()`: 127ms
- `buffer()`: 192ms
- `getStreamAsArrayBuffer()`: 129ms
- `arrayBuffer()`: 195ms
- `getStreamAsArray()`: 89ms
- `stream.toArray()`: 90ms
### Web ReadableStream (100 MB, binary)
- `getStream()`: 223ms
- `text()`: 221ms
- `getStreamAsBuffer()`: 182ms
- `buffer()`: 153ms
- `getStreamAsArrayBuffer()`: 171ms
- `arrayBuffer()`: 155ms
- `getStreamAsArray()`: 83ms
### Web ReadableStream (100 MB, text)
- `getStream()`: 141ms
- `text()`: 139ms
- `getStreamAsBuffer()`: 91ms
- `buffer()`: 80ms
- `getStreamAsArrayBuffer()`: 89ms
- `arrayBuffer()`: 81ms
- `getStreamAsArray()`: 21ms
[Benchmarks' source file](benchmarks/index.js).
## FAQ
### How is this different from [`concat-stream`](https://github.com/maxogden/concat-stream)?
This module accepts a stream instead of being one and returns a promise instead of using a callback. The API is simpler and it only supports returning a string, `Buffer`, an `ArrayBuffer` or an array. It doesn't have a fragile type inference. You explicitly choose what you want. And it doesn't depend on the huge `readable-stream` package.
## Related
- [get-stdin](https://github.com/sindresorhus/get-stdin) - Get stdin as a string or buffer
- [into-stream](https://github.com/sindresorhus/into-stream) - The opposite of this package