https://github.com/viccon/node-stream-chunker
A minimal node transform stream for splitting data into chunks
https://github.com/viccon/node-stream-chunker
javascript node node-js nodejs
Last synced: 4 months ago
JSON representation
A minimal node transform stream for splitting data into chunks
- Host: GitHub
- URL: https://github.com/viccon/node-stream-chunker
- Owner: viccon
- Created: 2019-12-02T12:14:33.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2023-01-05T02:07:44.000Z (over 3 years ago)
- Last Synced: 2025-10-13T08:40:46.904Z (8 months ago)
- Topics: javascript, node, node-js, nodejs
- Language: JavaScript
- Homepage:
- Size: 355 KB
- Stars: 4
- Watchers: 0
- Forks: 0
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Stream chunker
This module implements the Node transform stream interface. It keeps an internal
buffer of a given size, and will spit out a chunk each time it fills up.
If you want to split a large file into chunks, based on a delimiter, you would
basically do this:
``` javascript
const fs = require('fs');
const StreamChunker = require('./streamChunker');
const chunker = new StreamChunker(20, '\n'); // The delimiter used here is \n
fs.createReadStream('largeDatabaseFile.csv')
.pipe(chunker)
.on('data', data => {
// data here will be a chunk of 20 items.
})
.on('finish', () => console.log('Stream finished.'));
```
## Options
The StreamChunker constructor takes three arguments:
- chunkSize (number)
- delimiter (string)
- defaultEncoding (string, defaults to 'utf-8')