{"id":13525628,"url":"https://github.com/pierrec/node-lz4","last_synced_at":"2025-10-22T13:36:42.166Z","repository":{"id":3886977,"uuid":"4974131","full_name":"pierrec/node-lz4","owner":"pierrec","description":"LZ4 fast compression algorithm for NodeJS","archived":false,"fork":false,"pushed_at":"2024-04-03T16:29:40.000Z","size":20115,"stargazers_count":435,"open_issues_count":36,"forks_count":94,"subscribers_count":24,"default_branch":"master","last_synced_at":"2024-05-22T04:04:24.463Z","etag":null,"topics":["javascript","js","lz4","lz4-frame"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pierrec.png","metadata":{"files":{"readme":"README.md","changelog":"History.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2012-07-10T13:12:27.000Z","updated_at":"2024-06-04T13:56:43.060Z","dependencies_parsed_at":"2024-06-04T13:56:41.287Z","dependency_job_id":"4d679bac-930e-4928-8fa0-ca9876e6ed67","html_url":"https://github.com/pierrec/node-lz4","commit_stats":{"total_commits":122,"total_committers":23,"mean_commits":5.304347826086956,"dds":"0.33606557377049184","last_synced_commit":"943e768691a9fb29e2d9a88f9942463765a45468"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pierrec%2Fnode-lz4","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pierrec%2Fnode-lz4/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pierrec%2Fnode-lz4/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pierrec%2Fnode-lz4/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pierrec","download_url":"https://codeload.github.com/pierrec/node-lz4/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246591403,"owners_count":20801983,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["javascript","js","lz4","lz4-frame"],"created_at":"2024-08-01T06:01:20.560Z","updated_at":"2025-10-22T13:36:36.790Z","avatar_url":"https://github.com/pierrec.png","language":"JavaScript","readme":"# LZ4\n\n[LZ4](http://fastcompression.blogspot.fr/) is a very fast compression and decompression algorithm. This nodejs module provides a Javascript implementation of the decoder as well as native bindings to the LZ4 functions. Nodejs Streams are also supported for compression and decompression.\n\nNB.\nVersion 0.2 does not support the legacy format, only the one as of \"LZ4 Streaming Format 1.4\". Use version 0.1 if required.\n\n## Build\n\nWith NodeJS:\n\n```shell\ngit clone https://github.com/pierrec/node-lz4.git\ncd node-lz4\ngit submodule update --init --recursive\nnpm install\n```\n\n## Install\n\nWith NodeJS:\n\n```shell\nnpm install lz4\n```\n\nWithin the browser, using `build/lz4.js`:\n\n```html\n\u003cscript type=\"text/javascript\" src=\"/path/to/lz4.js\"\u003e\u003c/script\u003e\n\u003cscript type=\"text/javascript\"\u003e\n// Nodejs-like Buffer built-in\nvar Buffer = require('buffer').Buffer\nvar LZ4 = require('lz4')\n\n// Some data to be compressed\nvar data = 'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.'\ndata += data\n// LZ4 can only work on Buffers\nvar input = Buffer.from(data)\n// Initialize the output buffer to its maximum length based on the input data\nvar output = Buffer.alloc( LZ4.encodeBound(input.length) )\n\n// block compression (no archive format)\nvar compressedSize = LZ4.encodeBlock(input, output)\n// remove unnecessary bytes\noutput = output.slice(0, compressedSize)\n\nconsole.log( \"compressed data\", output )\n\n// block decompression (no archive format)\nvar uncompressed = Buffer.alloc(input.length)\nvar uncompressedSize = LZ4.decodeBlock(output, uncompressed)\nuncompressed = uncompressed.slice(0, uncompressedSize)\n\nconsole.log( \"uncompressed data\", uncompressed )\n\u003c/script\u003e\n```\n\n\nFrom github cloning, after having made sure that node and node-gyp are properly installed:\n\n```shell\nnpm i\nnode-gyp rebuild\n```\n\nSee below for more LZ4 functions.\n\n\n## Usage\n\n### Encoding\n\nThere are 2 ways to encode:\n\n* __asynchronous__ using nodejs Streams - slowest but can handle very large data sets (no memory limitations).\n* __synchronous__ by feeding the whole set of data - faster but is limited by the amount of memory\n\n\n#### Asynchronous encoding\n\nFirst, create an LZ4 encoding NodeJS stream with `LZ4#createEncoderStream(options)`.\n\n* `options` (_Object_): LZ4 stream options (optional)\n\t* `options.blockMaxSize` (_Number_): chunk size to use (default=4Mb)\n\t* `options.highCompression` (_Boolean_): use high compression (default=false)\n\t* `options.blockIndependence` (_Boolean_): (default=true)\n\t* `options.blockChecksum` (_Boolean_): add compressed blocks checksum (default=false)\n\t* `options.streamSize` (_Boolean_): add full LZ4 stream size (default=false)\n\t* `options.streamChecksum` (_Boolean_): add full LZ4 stream checksum (default=true)\n\t* `options.dict` (_Boolean_): use dictionary (default=false)\n\t* `options.dictId` (_Integer_): dictionary id (default=0)\n\n\nThe stream can then encode any data piped to it. It will emit a `data` event on each encoded chunk, which can be saved into an output stream.\n\nThe following example shows how to encode a file `test` into `test.lz4`.\n\n\n```javascript\nvar fs = require('fs')\nvar lz4 = require('lz4')\n\nvar encoder = lz4.createEncoderStream()\n\nvar input = fs.createReadStream('test')\nvar output = fs.createWriteStream('test.lz4')\n\ninput.pipe(encoder).pipe(output)\n```\n\n#### Synchronous encoding\n\nRead the data into memory and feed it to `LZ4#encode(input[, options])` to decode an LZ4 stream.\n\n* `input` (_Buffer_): data to encode\n* `options` (_Object_): LZ4 stream options (optional)\n\t* `options.blockMaxSize` (_Number_): chunk size to use (default=4Mb)\n\t* `options.highCompression` (_Boolean_): use high compression (default=false)\n\t* `options.blockIndependence` (_Boolean_): (default=true)\n\t* `options.blockChecksum` (_Boolean_): add compressed blocks checksum (default=false)\n\t* `options.streamSize` (_Boolean_): add full LZ4 stream size (default=false)\n\t* `options.streamChecksum` (_Boolean_): add full LZ4 stream checksum (default=true)\n\t* `options.dict` (_Boolean_): use dictionary (default=false)\n\t* `options.dictId` (_Integer_): dictionary id (default=0)\n\n\n```javascript\nvar fs = require('fs')\nvar lz4 = require('lz4')\n\nvar input = fs.readFileSync('test')\nvar output = lz4.encode(input)\n\nfs.writeFileSync('test.lz4', output)\n```\n\n\n### Decoding\n\nThere are 2 ways to decode:\n\n* __asynchronous__ using nodejs Streams - slowest but can handle very large data sets (no memory limitations)\n* __synchronous__ by feeding the whole LZ4 data - faster but is limited by the amount of memory\n\n\n#### Asynchronous decoding\n\nFirst, create an LZ4 decoding NodeJS stream with `LZ4#createDecoderStream()`.\n\n\nThe stream can then decode any data piped to it. It will emit a `data` event on each decoded sequence, which can be saved into an output stream.\n\nThe following example shows how to decode an LZ4 compressed file `test.lz4` into `test`.\n\n\n```javascript\nvar fs = require('fs')\nvar lz4 = require('lz4')\n\nvar decoder = lz4.createDecoderStream()\n\nvar input = fs.createReadStream('test.lz4')\nvar output = fs.createWriteStream('test')\n\ninput.pipe(decoder).pipe(output)\n```\n\n#### Synchronous decoding\n\nRead the data into memory and feed it to `LZ4#decode(input)` to produce an LZ4 stream.\n\n* `input` (_Buffer_): data to decode\n\n\n```javascript\nvar fs = require('fs')\nvar lz4 = require('lz4')\n\nvar input = fs.readFileSync('test.lz4')\nvar output = lz4.decode(input)\n\nfs.writeFileSync('test', output)\n```\n\n## Block level encoding/decoding\n\nIn some cases, it is useful to be able to manipulate an LZ4 block instead of an LZ4 stream. The functions to decode and encode are therefore exposed as:\n\n* `LZ4#decodeBlock(input, output[, startIdx, endIdx])` (_Number_) \u003e=0: uncompressed size, \u003c0: error at offset\n\t* `input` (_Buffer_): data block to decode\n\t* `output` (_Buffer_): decoded data block\n\t* `startIdx` (_Number_): input buffer start index (optional, default=0)\n\t* `endIdx` (_Number_): input buffer end index (optional, default=startIdx + input.length)\n* `LZ4#encodeBound(inputSize)` (_Number_): maximum size for a compressed block\n\t* `inputSize` (_Number_) size of the input, 0 if too large\n\tThis is required to size the buffer for a block encoded data\n* `LZ4#encodeBlock(input, output[, startIdx, endIdx])` (_Number_) \u003e0: compressed size, =0: not compressible\n\t* `input` (_Buffer_): data block to encode\n\t* `output` (_Buffer_): encoded data block\n\t* `startIdx` (_Number_): output buffer start index (optional, default=0)\n\t* `endIdx` (_Number_): output buffer end index (optional, default=startIdx + output.length)\n* `LZ4#encodeBlockHC(input, output[, compressionLevel])` (_Number_) \u003e0: compressed size, =0: not compressible\n\t* `input` (_Buffer_): data block to encode with high compression\n\t* `output` (_Buffer_): encoded data block\n\t* `compressionLevel` (_Number_): compression level between 3 and 12 (optional, default=9)\n\n\nBlocks do not have any magic number and are provided as is. It is useful to store somewhere the size of the original input for decoding.\nLZ4#encodeBlockHC() is not available as pure Javascript.\n\n\n## How it works\n\n* [LZ4 stream format](http://fastcompression.blogspot.fr/2011/05/lz4-explained.html)\n\n## Restrictions / Issues\n\n* `blockIndependence` property only supported for `true`\n\n\n## License\n\nMIT\n","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpierrec%2Fnode-lz4","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpierrec%2Fnode-lz4","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpierrec%2Fnode-lz4/lists"}