https://github.com/freeeve/bufr
a compressed/cached Buffer() wrapper for javascript
https://github.com/freeeve/bufr
buffer cache compress compressed javascript typescript
Last synced: 6 months ago
JSON representation
a compressed/cached Buffer() wrapper for javascript
- Host: GitHub
- URL: https://github.com/freeeve/bufr
- Owner: freeeve
- License: mit
- Created: 2019-01-13T15:31:49.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2023-01-04T21:36:55.000Z (over 3 years ago)
- Last Synced: 2025-03-08T07:22:40.891Z (over 1 year ago)
- Topics: buffer, cache, compress, compressed, javascript, typescript
- Language: TypeScript
- Homepage:
- Size: 1.67 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 17
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: code-of-conduct.md
Awesome Lists containing this project
README
# Bufr
[](https://travis-ci.com/freeeve/bufr)
[](https://coveralls.io/github/freeeve/bufr)
Bufr is a wrapper for Buffer, adding features like
auto-extend, compression, and a concept of an LRU cache--the
amount of memory designated to stay uncompressed for use.
It is not meant to improve performance, but it is meant to save memory,
for use when you can sacrifice a little performance for memory.
In testing, I found there is a balance between block size and
how random your access is. Smaller block size decompresses faster,
so it ends up being faster for small random reads. Larger block size
has less overhead, so if you're just appending and reading large chunks,
it is a fair bit faster.
## Usage
```typescript
const bufr = new Bufr();
const data = Buffer.from('hello');
bufr.writeBuffer(data, 0);
console.log(bufr.subBuffer(0, 0 + data.length).toString()); // hello
console.log(bufr.length); // 5
// append a 32-bit int
bufr.writeUInt32LE(123123, bufr.length);
console.log(bufr.length); // 9
```
### Specifying allocation size and cache size
Cache size should be a multiple of allocation size, because it will
end up being one, anyway (rounded down).
Cache size can be 0, but only in extreme cases.
Every read/write will decompress/compress the block.
```typescript
// default is 4 and 64 for allocSizeKb and cacheSizeKb, respectively
// this bufr will have at most 4MB uncompressed blocks at once,
// feel free to write as much as you want...
const bufr = new Bufr({allocSizeKb:128, cacheSizeKb:1024 * 4});
for(let i = 0; i < 1000000; i++) {
bufr.writeBuffer(Buffer.from('hello'.repeat(20)), bufr.length);
}
console.log(bufr.uncompressedSize); // 4194304
```
### Auto-extending
```typescript
// default is 4 and 64 for allocSizeKb and cacheSizeKb, respectively
// this bufr will have at most 4MB uncompressed blocks at once,
// feel free to write as much as you want...
// or start in the middle!
const bufr = new Bufr();
bufr.writeUInt32LE(123, 100000);
console.log(bufr.length); // 100004
console.log(bufr.uncompressedSize); // 65536 (64KB default cache size)
console.log(bufr.memoryUsage().compressed); // 633 (those 0 bytes compress well!)
```
## TODO
* Read/write to file
* Offer an API for inserting data (extending in the middle of a buffer without overwriting)
* Improve LRU caching performance
* Decrease memory used by metadata (block management)
* Offer more compression algorithms, namely I think snappy would be a good addition (currently using pako `inflateRaw`/`deflateRaw`)