https://github.com/regular/loner
A transform stream for node and browserify that ensures that certain data sequences appear in their own, separate chunks
https://github.com/regular/loner
javascript streaming streaming-parser streams
Last synced: 4 months ago
JSON representation
A transform stream for node and browserify that ensures that certain data sequences appear in their own, separate chunks
- Host: GitHub
- URL: https://github.com/regular/loner
- Owner: regular
- Created: 2017-02-07T13:24:53.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2017-02-07T15:01:08.000Z (over 8 years ago)
- Last Synced: 2025-01-31T16:18:24.931Z (4 months ago)
- Topics: javascript, streaming, streaming-parser, streams
- Language: JavaScript
- Size: 3.91 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
loner
===
A transform stream for node and browserify that ensures that certain data sequences appear in their own, separate chunks
When progressively parsing streaming binary data, you often need to find magic sequences of bytes that act as separators or "keywords". This can be surprisingly hard to implement efficiently (i.e. without a lot of buffering and copying data around) because the magic sequences might be spread across an arbitrary amount of chunks and you must take care to implement all those edge cases correctly.
`loner` does that for you. It ensures that each magic sequence you specify will be in its own data chunk, while otherwise leaving the sequence of chunks as it is. Thus all you need to do downstream is to compare each incoming chunk against the sequence you are looking for, for example by using [buffer-equal](https://www.npmjs.com/package/buffer-equal).
All the heavy lifting is actually done by [binary-stream-replace](https://www.npmjs.com/package/binary-stream-replace).
## Example
```
const loner = require('loner');
let PNGSignature = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
let l = loner(PNGSignature);
l.on('data', (data) => console.log(data) );
l.write(Buffer.from('Hello\x89\x50'));
l.write(Buffer.from([0x4e, 0x47, 0x0d, 0x0a]));
l.write(Buffer.from("\x1a\x0a\nWorld"));
l.end();
// output =>
//
//
//
```
## API
```
loner(seq1, seq2, ..., seqN)
```
*seq1* .. *seqN* Buffer objects containing sequences that you want isolated (on separate chunks). You can also pass Strings instead of Buffers, they will be converted internally.
**NOTE** If a sequence is fully contained within another seqeunce, as in `loner('Hello','lo')`, interesting things happen! The outcome then depends on the ordering of arguments. In this example, you'd get all the `lo`s separated, which obviously breaks each `hello` into `hel` and `lo`.