{"id":15295136,"url":"https://github.com/shampoofactory/lzfse_rust","last_synced_at":"2025-04-13T15:58:07.321Z","repository":{"id":57635749,"uuid":"355252824","full_name":"shampoofactory/lzfse_rust","owner":"shampoofactory","description":"A pure Rust LZFSE implementation.","archived":false,"fork":false,"pushed_at":"2023-04-06T14:07:42.000Z","size":3408,"stargazers_count":8,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-27T06:51:19.342Z","etag":null,"topics":["compression","compression-library","decompression","decompression-library","lzfse","rust","rust-lang"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shampoofactory.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-06T16:12:16.000Z","updated_at":"2025-03-20T16:01:58.000Z","dependencies_parsed_at":"2024-10-14T23:21:10.142Z","dependency_job_id":"ec71bb81-7900-4581-8ddd-49dbe76bf628","html_url":"https://github.com/shampoofactory/lzfse_rust","commit_stats":{"total_commits":121,"total_committers":2,"mean_commits":60.5,"dds":0.03305785123966942,"last_synced_commit":"e4e932f175a3062f7eb6e203fe6dcb39aa6242fd"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shampoofactory%2Flzfse_rust","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shampoofactory%2Flzfse_rust/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shampoofactory%2Flzfse_rust/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shampoofactory%2Flzfse_rust/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shampoofactory","download_url":"https://codeload.github.com/shampoofactory/lzfse_rust/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248741196,"owners_count":21154252,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compression","compression-library","decompression","decompression-library","lzfse","rust","rust-lang"],"created_at":"2024-09-30T17:08:44.524Z","updated_at":"2025-04-13T15:58:07.300Z","avatar_url":"https://github.com/shampoofactory.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# lzfse_rust\nRust LZFSE implementation.\n\n\n## Documentation\n\nhttps://docs.rs/lzfse_rust\n\n## Install\n\nSimply configure your `Cargo.toml`:\n\n```toml\n[dependencies]\nlzfse_rust = \"0.2\"\n```\n\n\n## Overview\n\nThis crate provides two LZFSE engines: one operating over user supplied memory buffers, and one operating over internal ring buffers.\n\nThe memory buffered engine works directly with input and output buffers that we supply.\nIt is exposed via `LzfseEncoder` and `LzfseDecoder` objects.\nWe would consider this engine when operating on `\u0026[u8]` and `Vec\u003cu8\u003e` objects.\n\nThe ring buffered engine works by streaming data in and out of it's internal ring buffers.\nIt is exposed via `LzfseRingEncoder` and `LzfseRingDecoder` objects.\nWe would consider this engine when operating on IO streams, or when we want to expose a `Read` or `Write` interface.\n\nCheck the documentation for additional information and examples.\n\n\n## Examples\n\nThis program compresses data from `stdin` into `stdout`. This example can be found in\n `examples/compress_ring.rs`\n\n```rust\nuse lzfse_rust::LzfseRingEncoder;\nuse std::io;\n\nfn main() -\u003e io::Result\u003c()\u003e {\n    let mut rdr = io::stdin();\n    let mut wtr = io::stdout();\n    let mut encoder = LzfseRingEncoder::default();\n    encoder.encode(\u0026mut rdr, \u0026mut wtr)?;\n    Ok(())\n}\n\n```\n\nThis program decompresses data from `stdin` into `stdout`. This example can be found in\n `examples/decompress_ring.rs`\n\n```rust\nuse lzfse_rust::LzfseRingDecoder;\nuse std::io;\n\nfn main() -\u003e io::Result\u003c()\u003e {\n    let mut rdr = io::stdin();\n    let mut wtr = io::stdout();\n    let mut decoder = LzfseRingDecoder::default();\n    decoder.decode(\u0026mut rdr, \u0026mut wtr)?;\n    Ok(())\n}\n\n```\n\n\n# Command line tool: lzfoo\n\nA fast, memory efficient and stream capable [lzfse](https://github.com/lzfse/lzfse) command line tool clone.\nPowered by [lzfse_rust](https://github.com/shampoofactory/lzfse_rust).\n\nInstall.\n\n```\n$ cargo install lzfoo\n```\n\nCompress `a.txt` to `a.txt.lzfse`:\n```\n$ lzfoo -encode -i a.txt -o a.txt.lzfse\n```\n\nCompress with stdin/ stdout:\n```\n$ lzfoo -encode -i \u003c a.txt \u003e a.txt.lzfse\n```\n```\n$ echo \"semper fidelis\" | lzfoo -encode \u003e a.txt.lzfse\n```\n\nDecompress `a.txt.lzfse` to `a.txt`:\n```\n$ lzfoo -decode -i a.txt.lzfse -o a.txt\n```\n\nDecompress with stdin/ stdout:\n```\n$ lzfoo -decode -i \u003c a.txt.lzfse \u003e a.txt\n```\n\nCheck the [lzfoo crate](https://github.com/shampoofactory/lzfse_rust/tree/main/lzfoo) for details.\n\n\n## Testing\n\nThis crate comes with a comprehensive test suite that is divided into unit tests and integration tests.\nThe unit tests check that the library's internal components are working, whilst the integration tests check that library as a whole is working.\nExtended tests are available but may take hours to complete.\n\nUnit tests:\n\n```\n$ cargo test\n```\n\nUnit tests, extended:\n\n```\n$ cargo test -- --ignored\n```\n\nIntegration tests:\n\n```\n$ cargo test --manifest-path test/Cargo.toml\n```\n\nIntegration tests, extended:\n\n```\n$ cargo test --manifest-path test/Cargo.toml -- --ignored\n```\n\nAdditional integration tests are available.\nNotably, validation tests with the reference LZFSE implementation.\nThese are described in [test](https://github.com/shampoofactory/lzfse_rust/tree/main/test), along with instructions on how to build and run them.\n\n\n## Performance\n\nAs a library and with the stated test machine, `lzfse_rust` outperforms `lzfse_ref`. However results will vary with different machines.\n\nThe benchmarks are powered with [Criterion](https://github.com/bheisler/criterion.rs) and formatted with [critcmp](https://github.com/BurntSushi/critcmp).\nThe machine is an 8GB Intel i5-2500K running [Ubuntu](https://ubuntu.com/) 18.04 (64 bit).\nThe dataset is taken from [Snappy](https://github.com/google/snappy).\nThe benchmark source is [here](https://github.com/shampoofactory/lzfse_rust/tree/main/bench), along with instructions on how to build and run the benchmarks.\n\n```\ngroup                          new/lzfse_ref/                         new/rust/                              new/rust_ring/\n-----                          --------------                         ---------                              --------------\ndecode/snap_uflat00_html       1.24    119.6±0.11µs   816.8 MB/sec    1.07    103.3±0.03µs   945.7 MB/sec    1.00     96.2±0.04µs  1014.9 MB/sec\ndecode/snap_uflat01_urls       1.18   1407.8±3.84µs   475.6 MB/sec    1.02   1211.8±0.11µs   552.5 MB/sec    1.00   1193.0±3.27µs   561.2 MB/sec\ndecode/snap_uflat02_jpg        1.07    353.7±0.05µs   331.9 MB/sec    1.00    330.7±0.91µs   355.0 MB/sec    1.02    336.1±0.85µs   349.3 MB/sec\ndecode/snap_uflat04_pdf        1.07    243.6±0.07µs   400.8 MB/sec    1.00    227.7±0.04µs   429.0 MB/sec    1.00    228.7±0.04µs   427.0 MB/sec\ndecode/snap_uflat05_html4      1.13    140.1±0.08µs     2.7 GB/sec    1.00    123.4±0.36µs     3.1 GB/sec    1.12    138.4±0.04µs     2.8 GB/sec\ndecode/snap_uflat06_txt1       1.22    469.4±0.05µs   309.0 MB/sec    1.09    420.8±1.21µs   344.7 MB/sec    1.00    384.3±0.05µs   377.4 MB/sec\ndecode/snap_uflat07_txt2       1.20    410.2±1.18µs   291.0 MB/sec    1.10    373.4±0.01µs   319.7 MB/sec    1.00    340.9±0.02µs   350.2 MB/sec\ndecode/snap_uflat08_txt3       1.25   1255.5±0.12µs   324.2 MB/sec    1.09   1096.9±3.32µs   371.0 MB/sec    1.00   1006.9±0.15µs   404.2 MB/sec\ndecode/snap_uflat09_txt4       1.18   1628.8±0.25µs   282.1 MB/sec    1.10   1511.5±3.03µs   304.0 MB/sec    1.00   1376.4±0.11µs   333.9 MB/sec\ndecode/snap_uflat10_pb         1.17    101.7±0.04µs  1112.3 MB/sec    1.04     90.2±0.03µs  1254.1 MB/sec    1.00     86.7±0.04µs  1304.3 MB/sec\ndecode/snap_uflat11_gaviota    1.28    486.0±0.05µs   361.7 MB/sec    1.09    413.2±0.05µs   425.4 MB/sec    1.00    379.5±0.03µs   463.1 MB/sec\nencode/snap_uflat00_html       1.83   1500.0±3.76µs    65.1 MB/sec    1.00    821.1±0.09µs   118.9 MB/sec    1.10    905.9±0.11µs   107.8 MB/sec\nencode/snap_uflat01_urls       1.45     13.1±0.00ms    51.3 MB/sec    1.00      9.0±0.00ms    74.2 MB/sec    1.01      9.1±0.00ms    73.8 MB/sec\nencode/snap_uflat02_jpg        1.11      2.1±0.01ms    55.4 MB/sec    1.00   1910.0±5.33µs    61.5 MB/sec    1.12      2.1±0.00ms    54.7 MB/sec\nencode/snap_uflat04_pdf        1.11   1695.1±0.19µs    57.6 MB/sec    1.00   1528.3±0.15µs    63.9 MB/sec    1.12   1705.8±0.16µs    57.3 MB/sec\nencode/snap_uflat05_html4      5.10      4.4±0.00ms    89.7 MB/sec    1.00    854.4±0.08µs   457.2 MB/sec    1.12    954.2±0.11µs   409.4 MB/sec\nencode/snap_uflat06_txt1       1.36      3.6±0.01ms    40.4 MB/sec    1.00      2.6±0.00ms    55.1 MB/sec    1.01      2.7±0.01ms    54.6 MB/sec\nencode/snap_uflat07_txt2       1.34      3.1±0.01ms    38.5 MB/sec    1.01      2.3±0.00ms    51.2 MB/sec    1.00      2.3±0.00ms    51.8 MB/sec\nencode/snap_uflat08_txt3       1.37      9.5±0.00ms    42.6 MB/sec    1.00      7.0±0.01ms    58.5 MB/sec    1.02      7.1±0.02ms    57.6 MB/sec\nencode/snap_uflat09_txt4       1.34     12.3±0.04ms    37.3 MB/sec    1.00      9.3±0.00ms    49.7 MB/sec    1.00      9.2±0.00ms    49.9 MB/sec\nencode/snap_uflat10_pb         1.95   1568.4±3.90µs    72.1 MB/sec    1.00    802.5±0.08µs   140.9 MB/sec    1.14    915.6±0.11µs   123.5 MB/sec\nencode/snap_uflat11_gaviota    1.49      3.5±0.00ms    50.2 MB/sec    1.00      2.4±0.00ms    74.8 MB/sec    1.01      2.4±0.00ms    73.9 MB/sec\n```\n\n```\nKey:\nlzfse_ref: lzfse reference library\nrust     : lzfse_rust\nrust_ring: lzfse_rust ring\n\nColumn: 1 2 3\n1: relative time                    lower is better, 1.00 is the fastest \n2: mean time ± standard deviation   lower is better\n3: throughput                       higher is better\n```\n\n## Minimum Rust version policy\n\nThis crate's minimum supported `rustc` version is `1.51.0`.\n\n\n## License\n\nLicensed under either of\n\n * Apache License, Version 2.0\n   ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n * MIT license\n   ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\n\nat your option.\n\n\n## Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in the work by you, as defined in the Apache-2.0 license, shall be\ndual licensed as above, without any additional terms or conditions.\n\n\n## Alternatives\n\n* [lzfse-rs](https://github.com/citruz/lzfse-rs) - bindings to the reference [LZFSE](https://github.com/lzfse/lzfse) implementation.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshampoofactory%2Flzfse_rust","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshampoofactory%2Flzfse_rust","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshampoofactory%2Flzfse_rust/lists"}