{"id":19138367,"url":"https://github.com/rochet2/lualzw","last_synced_at":"2026-07-29T04:00:22.900Z","repository":{"id":41086356,"uuid":"71663146","full_name":"Rochet2/lualzw","owner":"Rochet2","description":"A relatively fast LZW compression algorithm in pure lua","archived":false,"fork":false,"pushed_at":"2026-05-31T03:52:43.000Z","size":33,"stargazers_count":62,"open_issues_count":1,"forks_count":13,"subscribers_count":2,"default_branch":"master","last_synced_at":"2026-05-31T05:17:49.756Z","etag":null,"topics":["compression","encoding","lossless","lua","lzw-compression"],"latest_commit_sha":null,"homepage":null,"language":"Lua","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Rochet2.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-10-22T20:12:51.000Z","updated_at":"2026-05-25T11:04:33.000Z","dependencies_parsed_at":"2023-01-30T19:46:04.277Z","dependency_job_id":null,"html_url":"https://github.com/Rochet2/lualzw","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Rochet2/lualzw","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rochet2%2Flualzw","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rochet2%2Flualzw/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rochet2%2Flualzw/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rochet2%2Flualzw/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Rochet2","download_url":"https://codeload.github.com/Rochet2/lualzw/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rochet2%2Flualzw/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":36016154,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-29T02:00:04.910Z","response_time":95,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compression","encoding","lossless","lua","lzw-compression"],"created_at":"2024-11-09T06:42:45.326Z","updated_at":"2026-07-29T04:00:22.832Z","avatar_url":"https://github.com/Rochet2.png","language":"Lua","funding_links":[],"categories":[],"sub_categories":[],"readme":"# lualzw\n\nA relatively fast LZW compression algorithm in pure Lua.\n\n## Overview\n\nLossless compression for byte strings. The more repetition in the data, the better the ratio.\n\nThe library uses 16-bit dictionary codes (two bytes per code). The maximum dictionary size per level is 65280 codes.\n\nInput is processed as a sequence of **bytes** (Lua `string` semantics). UTF-8 text round-trips correctly because multibyte sequences are compressed as individual bytes, not as Unicode code points.\n\nWhile compressing, the algorithm checks whether the result would be strictly smaller than the input. If not, it returns an uncompressed passthrough instead (see [Wire format](#wire-format)).\n\n## Quick start\n\n```lua\nlocal lualzw = require(\"lualzw\")\n\nlocal input = \"foofoofoofoofoofoofoofoofoo\"\nlocal compressed = assert(lualzw.compress(input))\nlocal decompressed = assert(lualzw.decompress(compressed))\nassert(input == decompressed)\n```\n\n## Install\n\nCopy [`lualzw.lua`](lualzw.lua) onto your Lua `package.path`, or clone a tagged release:\n\n```sh\ngit clone --branch v1.1.0 --depth 1 https://github.com/Rochet2/lualzw.git\n```\n\n## Configuration\n\nCreate a custom codec with `configure()`. The default module (`require(\"lualzw\")`) uses `skip = {}` and control prefixes `u` / `c`.\n\n```lua\nlocal lualzw = require(\"lualzw\")\n\n-- Default: original encoding (may embed \\0 in dictionary codes)\nlocal legacy = lualzw.configure({ skip = {} })\n\n-- Preferred for null-free input when compressed codes must avoid \\0\nlocal nullsafe = lualzw.configure({ skip = { [0] = true } })\n```\n\n### Skipping null bytes in codes\n\nThe default skip list `{}` matches the original on-the-wire encoding. Dictionary codes are 16-bit byte pairs and **may include `\\0`**. That is fine for plain Lua strings, but problematic when compressed data passes through:\n\n- C APIs or bindings that treat `\\0` as end-of-string\n- Null-terminated storage or logging\n- Tools that truncate at the first null\n\nUse `{ skip = { [0] = true } }` so dictionary codes never use `0` as their **second** byte. For **null-free input**, compressed output then contains no `\\0`. If the input itself contains null bytes (or compression falls back to passthrough of such input), `\\0` can still appear as the first byte of a base code (`char(0, …)`). Both compressor and decompressor need the same `skip` setting; it is not stored in the payload.\n\nCustom control prefixes (both peers must match):\n\n```lua\nlocal custom = lualzw.configure({\n    skip = { [0] = true },\n    uncompressed = \"p\",\n    compressed = \"q\",\n})\n```\n\nSee [`configure(options)`](#configureoptions) for full option details.\n\n## Untrusted input\n\nlualzw is a **compression codec**, not encryption or authentication. Treat compressed data from untrusted sources as hostile.\n\nAlways call `decompress` with explicit limits:\n\n```lua\nlocal MAX = 64 * 1024\nlocal data, err = lualzw.decompress(payload, MAX, MAX * 2, MAX * 4)\nif not data then\n    -- reject message\nend\n```\n\n| Limit | Parameter | Protects against |\n| ----- | --------- | ---------------- |\n| Output size | `max_output_size` | Decompression bombs (huge expanded output) |\n| Input size | `max_input_size` | Large compressed blobs (memory / bandwidth) |\n| Dictionary steps | `max_codes` | CPU exhaustion during decode |\n\nBound `compress` on public endpoints as well:\n\n```lua\nlocal compressed, err = lualzw.compress(plaintext, MAX)\n```\n\n## API\n\nLoad the module:\n\n```lua\nlocal lualzw = require(\"lualzw\")\nprint(lualzw._VERSION) -- \"1.1.0\"\n```\n\nEach codec table (default, or from `configure()`) exports:\n\n| Member | Description |\n| ------ | ----------- |\n| `compress(input[, max_input_size])` | Compress a string |\n| `decompress(input[, max_output_size[, max_input_size[, max_codes]]])` | Decompress or passthrough |\n| `configure(options)` | Create a new codec; see [`configure(options)`](#configureoptions) |\n| `_VERSION` | Semantic version string |\n| `uncompressed` | Passthrough prefix byte for this codec |\n| `compressed` | Compressed prefix byte for this codec |\n\n### `configure(options)`\n\nReturns a **new codec table** with the same methods as the default module, using the given options. Options are fixed for the lifetime of that codec; they are not stored in compressed output, so both peers must use matching settings.\n\n**Parameters**\n\n| Name | Type | Default | Description |\n| ---- | ---- | ------- | ----------- |\n| `options` | `table` | `{}` | Configuration (all keys optional) |\n| `options.skip` | `table` | `{}` | Byte values that must not appear in dictionary codes (see below) |\n| `options.uncompressed` | `string` | `\"u\"` | One-byte prefix for passthrough output |\n| `options.compressed` | `string` | `\"c\"` | One-byte prefix for LZW output |\n\n**Returns**\n\nA codec table with `compress`, `decompress`, `configure`, `_VERSION`, `uncompressed`, and `compressed`.\n\n**`options.skip`**\n\nDictionary codes are 16-bit byte pairs. Skipped bytes never appear in those pairs (useful to keep compressed data free of `\\0`, etc.).\n\nTwo table forms are accepted:\n\n```lua\n-- Map form\n{ [0] = true, [1] = true }\n\n-- List form (byte values as array entries)\n{ 0, 1 }\n```\n\nEach skipped byte slightly reduces available dictionary codes. If too many bytes are skipped, `configure()` raises:\n\n```\ninvalid configuration, no character can be used in compression\n```\n\n**`options.uncompressed` / `options.compressed`**\n\n- Each must be a string of **exactly one byte**.\n- They must be **different** from each other.\n- Invalid values raise `invalid uncompressed control character`, `invalid compressed control character`, or `uncompressed and compressed control characters must differ`.\n\nChanging skip or control settings produces incompatible compressed data. Default `{}` skip matches the original master encoding.\n\n```lua\nlocal codec = lualzw.configure({\n    skip = { [0] = true },\n    uncompressed = \"u\",\n    compressed = \"c\",\n})\nprint(codec.uncompressed, codec.compressed) -- u    c\n```\n\n### `compress(input[, max_input_size])`\n\n**Returns:** compressed string, or `nil, error`.\n\n- Non-string input → `nil, \"string expected, got \u003ctype\u003e\"`\n- Bad limit type → `nil, \"number expected for max_input_size, got \u003ctype\u003e\"`\n- Invalid limit (negative, NaN, infinity) → `nil, \"invalid max_input_size\"`\n- Input longer than `max_input_size` → `nil, \"input exceeds limit\"`\n- Input of 0–1 bytes → passthrough (`u` prefix; see wire format)\n- Longer input → LZW compress if strictly smaller than input, otherwise passthrough\n- Internal failure → `nil, \"algorithm error, could not fetch word\"`\n\n### `decompress(input[, max_output_size[, max_input_size[, max_codes]]])`\n\n**Returns:** original string, or `nil, error`.\n\nAlways pass limits when decoding **untrusted** data (see [Untrusted input](#untrusted-input)).\n\n- Non-string input → `nil, \"string expected, got \u003ctype\u003e\"`\n- Invalid limit types → `nil, \"number expected for \u003cname\u003e, got \u003ctype\u003e\"`\n- Invalid limits (negative, NaN, infinity) → `nil, \"invalid \u003cname\u003e\"`\n- `#input` or body larger than `max_input_size` → `nil, \"compressed input exceeds limit\"`\n- Decompressed size exceeds `max_output_size` → `nil, \"decompressed output exceeds limit\"`\n- Dictionary growth exceeds `max_codes` → `nil, \"decompression step limit exceeded\"`\n- Invalid or corrupt payload → `nil, \"invalid input - not a compressed string\"` or `\"could not find last from dict. Invalid input?\"`\n\n## Wire format\n\nEach output starts with a one-byte control prefix configured on the codec (defaults shown):\n\n| Prefix | Meaning | Body |\n| ------ | ------- | ---- |\n| `u` (default) | Uncompressed passthrough | Original bytes |\n| `c` (default) | LZW compressed | Pairs of code bytes |\n\nExamples with default controls:\n\n| Input to `compress` | Output |\n| ------------------- | ------ |\n| `\"\"` | `\"u\"` |\n| `\"a\"` | `\"ua\"` |\n| Repetitive data | `\"c\" .. \u003ccode pairs\u003e` if smaller than input |\n| Incompressible data | `\"u\" .. input` |\n\n## Tests\n\n```sh\nlua spec/test.lua\n```\n\n## Benchmarks\n\nHistorical timings below were produced with `benchmark/profiling.lua`, which compares lualzw to [LibCompress](https://www.curseforge.com/wow/addons/libcompress). LibCompress is not bundled with this repo.\n\nFrom the repository root:\n\n```sh\nlua benchmark/profiling.lua\n```\n\nUse `--quick` for smaller inputs (10 000 bytes, 3 iterations):\n\n```sh\nlua benchmark/profiling.lua --quick\n```\n\nEach case runs the default and null-safe (`skip = { [0] = true }`) codecs. LibCompress is compared when installed.\n\n### Published results\n\nTimes are in seconds (average of 10 runs). Random inputs usually bail out to passthrough (100% of input size).\n\n**Input:** 1 000 000 random bytes\n\n| algorithm | compress | decompress | result % |\n| --------- | -------- | ---------- | -------- |\n| lualzw | 0.6622 | 0.0003 | 100 |\n| LibCompress | 2.1983 | 0.0024 | 100 |\n\n**Input:** 1 000 000 random ASCII bytes\n\n| algorithm | compress | decompress | result % |\n| --------- | -------- | ---------- | -------- |\n| lualzw | 0.812 | 0.0022 | 100 |\n| LibCompress | 1.782 | 0.0007 | 100 |\n\n**Input:** 1 000 000 repeating cycling bytes\n\n| algorithm | compress | decompress | result % |\n| --------- | -------- | ---------- | -------- |\n| lualzw | 0.3975 | 0.0262 | 4.5001 |\n| LibCompress | 0.3907 | 0.0264 | 6.6997 |\n\n**Input:** 1 000 000 identical bytes\n\n| algorithm | compress | decompress | result % |\n| --------- | -------- | ---------- | -------- |\n| lualzw | 0.7045 | 0.0026 | 0.2829 |\n| LibCompress | 0.6418 | 0.0038 | 0.4241 |\n\n**Input:** `\"ymn32h8hm8ekrwjkrn9f\"` × 50 000 (1 000 000 bytes)\n\n| algorithm | compress | decompress | result % |\n| --------- | -------- | ---------- | -------- |\n| lualzw | 0.4788 | 0.0088 | 1.2629 |\n| LibCompress | 0.4426 | 0.0093 | 1.8905 |\n\n## Error reference\n\n| Function | Condition | Error |\n| -------- | --------- | ----- |\n| `compress` | Wrong type | `\"string expected, got \u003ctype\u003e\"` |\n| `compress` | Bad limit type | `\"number expected for max_input_size, got \u003ctype\u003e\"` |\n| `compress` | Invalid limit | `\"invalid max_input_size\"` |\n| `compress` | Input too large | `\"input exceeds limit\"` |\n| `compress` | Internal | `\"algorithm error, could not fetch word\"` |\n| `decompress` | Wrong type | `\"string expected, got \u003ctype\u003e\"` |\n| `decompress` | Bad limit type | `\"number expected for \u003cname\u003e, got \u003ctype\u003e\"` |\n| `decompress` | Invalid limit | `\"invalid \u003cname\u003e\"` |\n| `decompress` | Empty / invalid | `\"invalid input - not a compressed string\"` |\n| `decompress` | Corrupt codes | `\"could not find last from dict. Invalid input?\"` |\n| `decompress` | Output limit | `\"decompressed output exceeds limit\"` |\n| `decompress` | Input limit | `\"compressed input exceeds limit\"` |\n| `decompress` | Step limit | `\"decompression step limit exceeded\"` |\n| `configure` | Bad control byte | `\"invalid uncompressed control character\"` / `\"invalid compressed control character\"` |\n| `configure` | Matching controls | `\"uncompressed and compressed control characters must differ\"` |\n| `configure` | Skip too aggressive | `\"invalid configuration, no character can be used in compression\"` |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frochet2%2Flualzw","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frochet2%2Flualzw","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frochet2%2Flualzw/lists"}