{"id":13527697,"url":"https://github.com/BurntSushi/rust-snappy","last_synced_at":"2025-04-01T09:32:00.469Z","repository":{"id":10160620,"uuid":"64718092","full_name":"BurntSushi/rust-snappy","owner":"BurntSushi","description":"Snappy compression implemented in Rust (including the Snappy frame format).","archived":false,"fork":false,"pushed_at":"2024-12-31T13:58:43.000Z","size":1233,"stargazers_count":469,"open_issues_count":8,"forks_count":45,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-27T11:05:25.309Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BurntSushi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["BurntSushi"]}},"created_at":"2016-08-02T02:41:48.000Z","updated_at":"2025-03-13T17:20:44.000Z","dependencies_parsed_at":"2024-01-13T22:53:27.920Z","dependency_job_id":"592f95e7-3643-4230-99f2-291f80fa40b4","html_url":"https://github.com/BurntSushi/rust-snappy","commit_stats":{"total_commits":130,"total_committers":14,"mean_commits":9.285714285714286,"dds":0.1461538461538462,"last_synced_commit":"d3a3e8c16663adf68d0bfd9f1a616fe4e392cd68"},"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurntSushi%2Frust-snappy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurntSushi%2Frust-snappy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurntSushi%2Frust-snappy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurntSushi%2Frust-snappy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BurntSushi","download_url":"https://codeload.github.com/BurntSushi/rust-snappy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246490814,"owners_count":20786064,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T06:01:57.288Z","updated_at":"2025-04-01T09:31:59.158Z","avatar_url":"https://github.com/BurntSushi.png","language":"Rust","funding_links":["https://github.com/sponsors/BurntSushi"],"categories":["Rust"],"sub_categories":[],"readme":"snap\n====\nA pure Rust implementation of the\n[Snappy compression algorithm](https://google.github.io/snappy/).\nIncludes streaming compression and decompression using the Snappy frame format.\nThis implementation is ported from both the\n[reference C++ implementation](https://github.com/google/snappy)\nand the\n[Go implementation](https://github.com/golang/snappy).\n\n[![Build status](https://github.com/BurntSushi/rust-snappy/workflows/ci/badge.svg)](https://github.com/BurntSushi/rust-snappy/actions)\n[![](https://meritbadge.herokuapp.com/snap)](https://crates.io/crates/snap)\n\nLicensed under the BSD 3-Clause.\n\n\n### Documentation\n\nhttps://docs.rs/snap\n\n\n### Usage\n\nAdd this to your `Cargo.toml`:\n\n```toml\n[dependencies]\nsnap = \"1\"\n```\n\n\n### Example: compress data on `stdin`\n\nThis program reads data from `stdin`, compresses it and emits it to `stdout`.\nThis example can be found in `examples/compress.rs`:\n\n```rust\nuse std::io;\n\nfn main() {\n    let stdin = io::stdin();\n    let stdout = io::stdout();\n\n    let mut rdr = stdin.lock();\n    // Wrap the stdout writer in a Snappy writer.\n    let mut wtr = snap::write::FrameEncoder::new(stdout.lock());\n    io::copy(\u0026mut rdr, \u0026mut wtr).expect(\"I/O operation failed\");\n}\n```\n\n\n### Example: decompress data on `stdin`\n\nThis program reads data from `stdin`, decompresses it and emits it to `stdout`.\nThis example can be found in `examples/decompress.rs`:\n\n```rust\nuse std::io;\n\nfn main() {\n    let stdin = io::stdin();\n    let stdout = io::stdout();\n\n    // Wrap the stdin reader in a Snappy reader.\n    let mut rdr = snap::read::FrameDecoder::new(stdin.lock());\n    let mut wtr = stdout.lock();\n    io::copy(\u0026mut rdr, \u0026mut wtr).expect(\"I/O operation failed\");\n}\n```\n\n\n### Example: the szip tool\n\n`szip` is a tool with similar behavior as `gzip`, except it uses Snappy\ncompression. It can be installed with Cargo:\n\n```\n$ cargo install szip\n```\n\nTo compress a file, run `szip file`. To decompress a file, run\n`szip -d file.sz`. See `szip --help` for more details.\n\n\n### Testing\n\nThis crate is tested against the reference C++ implementation of Snappy.\nCurrently, compression is byte-for-byte equivalent with the C++ implementation.\nThis seems like a reasonable starting point, although it is not necessarily\na goal to always maintain byte-for-byte equivalence.\n\nTests against the reference C++ implementation can be run with\n`cargo test --features cpp`. Note that you will need to have the C++ Snappy\nlibrary in your `LD_LIBRARY_PATH` (or equivalent).\n\nTo run tests, you'll need to explicitly run the `test` crate:\n\n```\n$ cargo test --manifest-path test/Cargo.toml\n```\n\nTo test that this library matches the output of the reference C++ library, use:\n\n```\n$ cargo test --manifest-path test/Cargo.toml --features cpp\n```\n\nTests are in a separate crate because of the dependency on the C++ reference\nlibrary. Namely, Cargo does not yet permit optional dev dependencies.\n\n\n### Minimum Rust version policy\n\nThis crate's minimum supported `rustc` version is `1.39.0`.\n\nThe current policy is that the minimum Rust version required to use this crate\ncan be increased in minor version updates. For example, if `crate 1.0` requires\nRust 1.20.0, then `crate 1.0.z` for all values of `z` will also require Rust\n1.20.0 or newer. However, `crate 1.y` for `y \u003e 0` may require a newer minimum\nversion of Rust.\n\nIn general, this crate will be conservative with respect to the minimum\nsupported version of Rust.\n\n\n### Performance\n\nThe performance of this implementation should roughly match the performance of\nthe C++ implementation on x86_64. Below are the results of the microbenchmarks\n(as defined in the C++ library):\n\n```\ngroup                         snappy/cpp/                            snappy/snap/\n-----                         -----------                            ------------\ncompress/zflat00_html         1.00     94.5±0.62µs  1033.1 MB/sec    1.02     96.1±0.74µs  1016.2 MB/sec\ncompress/zflat01_urls         1.00   1182.3±8.89µs   566.3 MB/sec    1.04  1235.3±11.99µs   542.0 MB/sec\ncompress/zflat02_jpg          1.00      7.2±0.11µs    15.9 GB/sec    1.01      7.3±0.06µs    15.8 GB/sec\ncompress/zflat03_jpg_200      1.10    262.4±1.84ns   727.0 MB/sec    1.00    237.5±2.95ns   803.2 MB/sec\ncompress/zflat04_pdf          1.02     10.3±0.18µs     9.2 GB/sec    1.00     10.1±0.16µs     9.4 GB/sec\ncompress/zflat05_html4        1.00    399.2±5.36µs   978.4 MB/sec    1.01    404.0±2.46µs   966.8 MB/sec\ncompress/zflat06_txt1         1.00    397.3±2.61µs   365.1 MB/sec    1.00    398.5±3.06µs   364.0 MB/sec\ncompress/zflat07_txt2         1.00    352.8±3.20µs   338.4 MB/sec    1.01    355.2±5.01µs   336.1 MB/sec\ncompress/zflat08_txt3         1.01   1058.8±6.85µs   384.4 MB/sec    1.00   1051.8±6.74µs   386.9 MB/sec\ncompress/zflat09_txt4         1.00   1444.1±8.10µs   318.2 MB/sec    1.00  1450.0±13.36µs   316.9 MB/sec\ncompress/zflat10_pb           1.00     85.1±0.58µs  1328.6 MB/sec    1.02     87.0±0.90µs  1300.2 MB/sec\ncompress/zflat11_gaviota      1.07    311.9±4.27µs   563.5 MB/sec    1.00    291.9±1.86µs   602.3 MB/sec\ndecompress/uflat00_html       1.03     36.9±0.28µs     2.6 GB/sec    1.00     36.0±0.25µs     2.7 GB/sec\ndecompress/uflat01_urls       1.04    437.4±2.89µs  1530.7 MB/sec    1.00    419.9±3.10µs  1594.6 MB/sec\ndecompress/uflat02_jpg        1.00      4.6±0.05µs    24.9 GB/sec    1.00      4.6±0.03µs    25.0 GB/sec\ndecompress/uflat03_jpg_200    1.08    122.4±1.06ns  1558.6 MB/sec    1.00    112.8±1.35ns  1690.8 MB/sec\ndecompress/uflat04_pdf        1.00      5.7±0.05µs    16.8 GB/sec    1.10      6.2±0.07µs    15.3 GB/sec\ndecompress/uflat05_html4      1.01    164.1±1.71µs     2.3 GB/sec    1.00    162.6±2.16µs     2.3 GB/sec\ndecompress/uflat06_txt1       1.08    146.6±1.01µs   989.5 MB/sec    1.00    135.3±1.11µs  1072.0 MB/sec\ndecompress/uflat07_txt2       1.09    130.2±0.93µs   916.6 MB/sec    1.00    119.2±0.96µs  1001.8 MB/sec\ndecompress/uflat08_txt3       1.07    387.2±2.30µs  1051.0 MB/sec    1.00    361.9±6.29µs  1124.7 MB/sec\ndecompress/uflat09_txt4       1.09    536.1±3.47µs   857.2 MB/sec    1.00    494.0±5.05µs   930.2 MB/sec\ndecompress/uflat10_pb         1.00     32.5±0.19µs     3.4 GB/sec    1.05     34.0±0.48µs     3.2 GB/sec\ndecompress/uflat11_gaviota    1.00    142.1±2.05µs  1236.7 MB/sec    1.00    141.5±0.92µs  1242.3 MB/sec\n```\n\nNotes: These benchmarks were run with Snappy/C++ 1.1.8. Both the C++ and Rust\nbenchmarks were run with the same benchmark harness. Benchmarks were run on an\nIntel i7-6900K.\n\nAdditionally, here are the benchmarks run on the same machine from the Go\nimplementation of Snappy (which has a hand rolled implementation in Assembly).\nNote that these were run using Go's microbenchmark tool, so the numbers may not\nbe directly comparable, but they should serve as a useful signpost:\n\n```\nBenchmark_UFlat0           25040             45180 ns/op        2266.49 MB/s\nBenchmark_UFlat1            2648            451475 ns/op        1555.10 MB/s\nBenchmark_UFlat2          229965              4788 ns/op        25709.01 MB/s\nBenchmark_UFlat3        11355555               101 ns/op        1973.65 MB/s\nBenchmark_UFlat4          196551              6055 ns/op        16912.64 MB/s\nBenchmark_UFlat5            6016            189219 ns/op        2164.68 MB/s\nBenchmark_UFlat6            6914            166371 ns/op         914.16 MB/s\nBenchmark_UFlat7            8173            142506 ns/op         878.41 MB/s\nBenchmark_UFlat8            2744            436424 ns/op         977.84 MB/s\nBenchmark_UFlat9            1999            591141 ns/op         815.14 MB/s\nBenchmark_UFlat10          28885             37291 ns/op        3180.04 MB/s\nBenchmark_UFlat11           7308            163366 ns/op        1128.26 MB/s\nBenchmark_ZFlat0           12902             91231 ns/op        1122.43 MB/s\nBenchmark_ZFlat1             997           1200579 ns/op         584.79 MB/s\nBenchmark_ZFlat2          136762              7832 ns/op        15716.53 MB/s\nBenchmark_ZFlat3         4896124               245 ns/op         817.27 MB/s\nBenchmark_ZFlat4          117643             10129 ns/op        10109.44 MB/s\nBenchmark_ZFlat5            2934            394742 ns/op        1037.64 MB/s\nBenchmark_ZFlat6            3008            382877 ns/op         397.23 MB/s\nBenchmark_ZFlat7            3411            344916 ns/op         362.93 MB/s\nBenchmark_ZFlat8             966           1057985 ns/op         403.36 MB/s\nBenchmark_ZFlat9             854           1429024 ns/op         337.20 MB/s\nBenchmark_ZFlat10          13861             83040 ns/op        1428.08 MB/s\nBenchmark_ZFlat11           4070            293952 ns/op         627.04 MB/s\n```\n\nTo run benchmarks, including the reference C++ implementation, do the\nfollowing:\n\n```\n$ cd bench\n$ cargo bench --features cpp -- --save-baseline snappy\n```\n\nTo compare them, as shown above, install\n[`critcmp`](https://github.com/BurntSushi/critcmp)\nand run (assuming you saved the baseline above under the name `snappy`):\n\n```\n$ critcmp snappy -g '.*?/(.*$)'\n```\n\nFinally, the Go benchmarks were run with the following command on commit\n`ff6b7dc8`:\n\n```\n$ go test -cpu 1 -bench Flat -download\n```\n\n\n### Comparison with other Snappy crates\n\n* `snappy` - These are bindings to the C++ library. No support for the Snappy\n  frame format.\n* `snappy_framed` - Implements the Snappy frame format on top of the `snappy`\n  crate.\n* `rsnappy` - Written in pure Rust, but lacks documentation and the Snappy\n  frame format. Performance is unclear and tests appear incomplete.\n* `snzip` - Was created and immediately yanked from crates.io.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBurntSushi%2Frust-snappy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FBurntSushi%2Frust-snappy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBurntSushi%2Frust-snappy/lists"}