{"id":21655305,"url":"https://github.com/pseitz/lz4_flex","last_synced_at":"2026-04-01T23:04:19.630Z","repository":{"id":39078124,"uuid":"240949621","full_name":"PSeitz/lz4_flex","owner":"PSeitz","description":"Fastest pure Rust implementation of LZ4 compression/decompression.","archived":false,"fork":false,"pushed_at":"2024-04-11T02:10:18.000Z","size":12564,"stargazers_count":408,"open_issues_count":10,"forks_count":30,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-04-14T09:48:40.926Z","etag":null,"topics":["compression","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PSeitz.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-16T19:11:41.000Z","updated_at":"2024-05-21T11:00:45.896Z","dependencies_parsed_at":"2024-01-11T05:28:48.434Z","dependency_job_id":"2e1263fd-e11c-4406-b268-66b9e08e91bf","html_url":"https://github.com/PSeitz/lz4_flex","commit_stats":{"total_commits":431,"total_committers":11,"mean_commits":39.18181818181818,"dds":"0.49651972157772617","last_synced_commit":"894b13c7478ce23ca093917f6d0d2caa3621ab12"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PSeitz%2Flz4_flex","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PSeitz%2Flz4_flex/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PSeitz%2Flz4_flex/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PSeitz%2Flz4_flex/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PSeitz","download_url":"https://codeload.github.com/PSeitz/lz4_flex/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248166861,"owners_count":21058481,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compression","rust"],"created_at":"2024-11-25T08:31:28.051Z","updated_at":"2026-04-01T23:04:19.623Z","avatar_url":"https://github.com/PSeitz.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Rust](https://github.com/PSeitz/lz4_flex/workflows/Rust/badge.svg)\n[![Docs](https://img.shields.io/docsrs/lz4_flex)](https://docs.rs/lz4_flex)\n[![Crates.io](https://img.shields.io/crates/v/lz4_flex.svg)](https://crates.io/crates/lz4_flex)\n\n# lz4_flex\n\n![lz4_flex_logo](https://raw.githubusercontent.com/PSeitz/lz4_flex/master/logo.jpg)\n\nFastest LZ4 implementation in Rust. Originally based on [redox-os' lz4 compression](https://crates.io/crates/lz4-compress), but now a complete rewrite.\nThe results in the table are from a benchmark in this project (66Kb JSON, 10MB dickens) with the block format. \n\nAMD Ryzen 7 5900HX, rustc 1.69.0 (84c898d65 2023-04-16), Manjaro, CPU Boost Disabled, CPU Governor: Performance\n\n66Kb JSON\n|    Compressor        | Compression | Decompression | Ratio\t\t |\n|----------------------|-------------|---------------|---------------|\n| lz4_flex unsafe w. unchecked_decode  | 1615 MiB/s   | 5973 MiB/s    | 0.2284   \t |\n| lz4_flex unsafe | 1615 MiB/s   | 5512 MiB/s    | 0.2284   \t |\n| lz4_flex safe        | 1272 MiB/s   | 4540 MiB/s    | 0.2284   \t |\n| lzzz (lz4 1.9.3)     | 1469 MiB/s   | 5313 MiB/s    | 0.2283   \t |\n| lz4_fear             | 662 MiB/s   | 939 MiB/s     | 0.2283\t     |\n| snap                 | 1452 MiB/s   | 1649 MiB/s     | 0.2242      |\n\n10 Mb dickens\n|    Compressor        | Compression | Decompression | Ratio\t\t |\n|----------------------|-------------|---------------|---------------|\n| lz4_flex unsafe w. unchecked_decode       | 347 MiB/s   | 3168 MiB/s    |  0.6372  \t |\n| lz4_flex unsafe      | 347 MiB/s   | 2734 MiB/s    |  0.6372  \t |\n| lz4_flex safe        | 259 MiB/s   | 2338 MiB/s    | 0.6372 |\n| lzzz (lz4 1.9.3)     | 324 MiB/s | 2759 MiB/s    | 0.6372 |\n| lz4_fear             | 201 MiB/s   | 370 MiB/s     | 0.6372 |\n| snap                 | 286 MiB/s   | 679 MiB/s     | 0.6276 |\n\n## Features\n- Very good logo\n- LZ4 Block format\n- LZ4 Frame format (thanks @arthurprs)\n- High performance\n- 1,5s clean release build time\n- Feature flags to configure safe/unsafe code usage\n- no-std support with block format (thanks @coolreader18)\n- 32-bit support\n\n## Usage: \nCompression and decompression uses no unsafe via the default feature flags \"safe-encode\" and \"safe-decode\". If you need more performance you can disable them (e.g. with no-default-features).\n\nSafe:\n```\nlz4_flex = { version = \"0.12\" }\n```\n\nPerformance:\n```\nlz4_flex = { version = \"0.12\", default-features = false }\n```\n\n### Block Format\nThe block format is only valid for smaller data chunks as block is de/compressed in memory.\nFor larger data use the frame format, which consists of multiple blocks.\n\n```rust\nuse lz4_flex::block::{compress_prepend_size, decompress_size_prepended};\n\nfn main(){\n    let input: \u0026[u8] = b\"Hello people, what's up?\";\n    let compressed = compress_prepend_size(input);\n    let uncompressed = decompress_size_prepended(\u0026compressed).unwrap();\n    assert_eq!(input, uncompressed);\n}\n```\n\n\n## no_std support\n\nno_std support is currently only for the block format, since the frame format uses `std::io::Write`, which is not available in core.\n\n## Benchmarks\nThe benchmark is run with criterion, the test files are in the benches folder.\n\nCurrently 4 implementations are compared, this one, [lz-fear](https://github.com/main--/rust-lz-fear), the [c version via rust bindings](https://crates.io/crates/lzzzz) and [snappy](https://github.com/burntsushi/rust-snappy). \nThe lz4-flex version is tested with the feature flags safe-decode and safe-encode switched on and off.\n\n- lz4_cpp: https://crates.io/crates/lzzzz\n- lz-fear: https://github.com/main--/rust-lz-fear\n- snap: https://github.com/burntsushi/rust-snappy \n\nTested on AMD Ryzen 7 5900HX, rustc 1.69.0 (84c898d65 2023-04-16), Manjaro, CPU Boost Disabled, CPU 3GHZ\n\n### Results v0.11.0 02-06-2023 (safe-decode and safe-encode off)\n`cargo bench --no-default-features`\n\n![Compress](./compress_bench.svg)\n\n![Decompress](./decompress_bench.svg)\n\n### Results v0.11.0 02-06-2023 (safe-decode and safe-encode on)\n`cargo bench`\n\n![Compress](./compress_bench_safe.svg)\n\n![Decompress](./decompress_bench_safe.svg)\n\n## Miri\n\n[Miri](https://github.com/rust-lang/miri) can be used to find issues related to incorrect unsafe usage:\n\n`MIRIFLAGS=\"-Zmiri-disable-isolation -Zmiri-disable-stacked-borrows\" cargo +nightly miri test --no-default-features --features frame`\n\n## Fuzzer\nThis fuzz target generates corrupted data for the decompressor:\\\n`cargo +nightly fuzz run fuzz_decomp_corrupt_block` and `cargo +nightly fuzz run fuzz_decomp_corrupt_frame`\n\nThis fuzz target asserts that the decompressor does not leak previous data when reusing an output buffer:\\\n`cargo +nightly fuzz run fuzz_decomp_no_output_leak`\n\nThis fuzz target asserts that a compression and decompression roundtrip returns the original input:\\\n`cargo +nightly fuzz run fuzz_roundtrip` and `cargo +nightly fuzz run fuzz_roundtrip_frame`\n\nThis fuzz target asserts compression with cpp and decompression with lz4_flex returns the original input:\\\n`cargo +nightly fuzz run fuzz_roundtrip_cpp_compress`\n\n## Bindings in other languages\n - Node.js: [lz4-napi](https://github.com/antoniomuso/lz4-napi) \n - Wasm: [lz4-wasm](https://github.com/PSeitz/lz4-wasm)\n - Python: [safelz4](https://github.com/LVivona/safelz4)\n\n## TODO\n- High compression\n\n## Migrate from v0.10 to v0.11.1\nTo migrate, just remove the `checked-decode` feature flag if you used it.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpseitz%2Flz4_flex","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpseitz%2Flz4_flex","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpseitz%2Flz4_flex/lists"}