{"id":14110283,"url":"https://github.com/rust-rse/reed-solomon-erasure","last_synced_at":"2025-12-12T16:44:33.404Z","repository":{"id":37541149,"uuid":"114504362","full_name":"rust-rse/reed-solomon-erasure","owner":"rust-rse","description":"[Looking for new owners/maintainers, see #88] Rust implementation of Reed-Solomon erasure coding","archived":false,"fork":false,"pushed_at":"2023-08-30T10:35:50.000Z","size":870,"stargazers_count":201,"open_issues_count":17,"forks_count":64,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-08-14T09:42:19.071Z","etag":null,"topics":["erasure-coding","reed-solomon","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rust-rse.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-12-17T02:37:08.000Z","updated_at":"2025-07-23T00:38:21.000Z","dependencies_parsed_at":"2024-01-08T07:57:38.673Z","dependency_job_id":"405d1464-91f6-45ee-8f9a-c24012792500","html_url":"https://github.com/rust-rse/reed-solomon-erasure","commit_stats":{"total_commits":832,"total_committers":17,"mean_commits":48.94117647058823,"dds":0.09495192307692313,"last_synced_commit":"ac2b561e406fedcccd144a436580921906d3eee9"},"previous_names":["darrenldl/reed-solomon-erasure"],"tags_count":22,"template":false,"template_full_name":null,"purl":"pkg:github/rust-rse/reed-solomon-erasure","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rust-rse%2Freed-solomon-erasure","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rust-rse%2Freed-solomon-erasure/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rust-rse%2Freed-solomon-erasure/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rust-rse%2Freed-solomon-erasure/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rust-rse","download_url":"https://codeload.github.com/rust-rse/reed-solomon-erasure/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rust-rse%2Freed-solomon-erasure/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270481293,"owners_count":24591328,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-14T02:00:10.309Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["erasure-coding","reed-solomon","rust"],"created_at":"2024-08-14T10:02:45.904Z","updated_at":"2025-12-12T16:44:33.345Z","avatar_url":"https://github.com/rust-rse.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"# reed-solomon-erasure\n[![Build Status](https://travis-ci.org/darrenldl/reed-solomon-erasure.svg?branch=master)](https://travis-ci.org/darrenldl/reed-solomon-erasure)\n[![Build status](https://ci.appveyor.com/api/projects/status/47c0emjoa9bhpjlb/branch/master?svg=true)](https://ci.appveyor.com/project/darrenldl/reed-solomon-erasure/branch/master)\n[![codecov](https://codecov.io/gh/darrenldl/reed-solomon-erasure/branch/master/graph/badge.svg)](https://codecov.io/gh/darrenldl/reed-solomon-erasure)\n[![Coverage Status](https://coveralls.io/repos/github/darrenldl/reed-solomon-erasure/badge.svg?branch=master)](https://coveralls.io/github/darrenldl/reed-solomon-erasure?branch=master)\n[![Crates](https://img.shields.io/crates/v/reed-solomon-erasure.svg)](https://crates.io/crates/reed-solomon-erasure)\n[![Documentation](https://docs.rs/reed-solomon-erasure/badge.svg)](https://docs.rs/reed-solomon-erasure)\n[![dependency status](https://deps.rs/repo/github/darrenldl/reed-solomon-erasure/status.svg)](https://deps.rs/repo/github/darrenldl/reed-solomon-erasure)\n\nRust implementation of Reed-Solomon erasure coding\n\nWASM builds are also available, see section **WASM usage** below for details\n\nThis is a port of [BackBlaze's Java implementation](https://github.com/Backblaze/JavaReedSolomon), [Klaus Post's Go implementation](https://github.com/klauspost/reedsolomon), and [Nicolas Trangez's Haskell implementation](https://github.com/NicolasT/reedsolomon).\n\nVersion `1.X.X` copies BackBlaze's implementation, and is less performant as there were fewer places where parallelism could be added.\n\nVersion `\u003e= 2.0.0` copies Klaus Post's implementation. The SIMD C code is copied from Nicolas Trangez's implementation with minor modifications.\n\nSee [Notes](#notes) and [License](#license) section for details.\n\n## WASM usage\n\nSee [here](wasm/README.md) for details\n\n## Rust usage\nAdd the following to your `Cargo.toml` for the normal version (pure Rust version)\n```toml\n[dependencies]\nreed-solomon-erasure = \"4.0\"\n```\nor the following for the version which tries to utilise SIMD\n```toml\n[dependencies]\nreed-solomon-erasure = { version = \"4.0\", features = [ \"simd-accel\" ] }\n```\nand the following to your crate root\n```rust\nextern crate reed_solomon_erasure;\n```\n\nNOTE: `simd-accel` is tuned for Haswell+ processors on x86-64 and not in any way for other architectures, set\nenvironment variable `RUST_REED_SOLOMON_ERASURE_ARCH` during build to force compilation of C code for specific architecture (`-march` flag in\nGCC/Clang). Even on x86-64 you can achieve better performance by setting it to `native`, but it will stop running on\nolder CPUs, YMMV.\n\n## Example\n```rust\n#[macro_use(shards)]\nextern crate reed_solomon_erasure;\n\nuse reed_solomon_erasure::galois_8::ReedSolomon;\n// or use the following for Galois 2^16 backend\n// use reed_solomon_erasure::galois_16::ReedSolomon;\n\nfn main () {\n    let r = ReedSolomon::new(3, 2).unwrap(); // 3 data shards, 2 parity shards\n\n    let mut master_copy = shards!(\n        [0, 1,  2,  3],\n        [4, 5,  6,  7],\n        [8, 9, 10, 11],\n        [0, 0,  0,  0], // last 2 rows are parity shards\n        [0, 0,  0,  0]\n    );\n\n    // Construct the parity shards\n    r.encode(\u0026mut master_copy).unwrap();\n\n    // Make a copy and transform it into option shards arrangement\n    // for feeding into reconstruct_shards\n    let mut shards: Vec\u003c_\u003e = master_copy.iter().cloned().map(Some).collect();\n\n    // We can remove up to 2 shards, which may be data or parity shards\n    shards[0] = None;\n    shards[4] = None;\n\n    // Try to reconstruct missing shards\n    r.reconstruct(\u0026mut shards).unwrap();\n\n    // Convert back to normal shard arrangement\n    let result: Vec\u003c_\u003e = shards.into_iter().filter_map(|x| x).collect();\n\n    assert!(r.verify(\u0026result).unwrap());\n    assert_eq!(master_copy, result);\n}\n```\n\n## Benchmark it yourself\nYou can test performance under different configurations quickly (e.g. data parity shards ratio, parallel parameters)\nwith standard `cargo bench` command.\n\n## Performance\nVersion `1.X.X`, `2.0.0` do not utilise SIMD.\n\nVersion `2.1.0` onward uses Nicolas's C files for SIMD operations.\n\nMachine: laptop with `Intel(R) Core(TM) i5-3337U CPU @ 1.80GHz (max 2.70GHz) 2 Cores 4 Threads`\n\nBelow shows the result of one of the test configurations, other configurations show similar results in terms of ratio.\n\n|Configuration| Klaus Post's | \u003e= 2.1.0 \u0026\u0026 \u003c 4.0.0 | 2.0.X | 1.X.X |\n|---|---|---|---|---|\n| 10x2x1M | ~7800MB/s |~4500MB/s | ~1000MB/s | ~240MB/s |\n\nVersions `\u003e= 4.0.0` have not been benchmarked thoroughly yet\n\n## Benchmarking\nYou can run benchmarks via `cargo bench`. To enable simd acceleration during benchmarks use `cargo bench --features simd-accel`.\n\n## Changelog\n[Changelog](CHANGELOG.md)\n\n## Contributions\nContributions are welcome. Note that by submitting contributions, you agree to license your work under the same license used by this project as stated in the LICENSE file.\n\n## Credits\n#### Library overhaul and Galois 2^16 backend\nMany thanks to the following people for overhaul of the library and introduction of Galois 2^16 backend\n\n  - [@drskalman](https://github.com/drskalman)\n\n  - Jeff Burdges [@burdges](https://github.com/burdges)\n\n  - Robert Habermeier [@rphmeier](https://github.com/rphmeier)\n\n#### WASM builds\nMany thanks to Nazar Mokrynskyi [@nazar-pc](https://github.com/nazar-pc) for submitting his package for WASM builds\n\nHe is the original author of the files stored in `wasm` folder. The files may have been modified by me later.\n\n#### AVX512 support\nMany thanks to [@sakridge](https://github.com/sakridge) for adding support for AVX512 (see [PR #69](https://github.com/darrenldl/reed-solomon-erasure/pull/69))\n\n#### build.rs improvements\nMany thanks to [@ryoqun](https://github.com/ryoqun) for improving the usability of the library in the context of cross-compilation (see [PR #75](https://github.com/darrenldl/reed-solomon-erasure/pull/75))\n\n#### no_std support\nMany thanks to Nazar Mokrynskyi [@nazar-pc](https://github.com/nazar-pc) for adding `no_std` support (see [PR #90](https://github.com/darrenldl/reed-solomon-erasure/pull/90))\n\n#### Testers\nMany thanks to the following people for testing and benchmarking on various platforms\n\n  - Laurențiu Nicola [@lnicola](https://github.com/lnicola/) (platforms: Linux, Intel)\n\n  - Roger Andersen [@hexjelly](https://github.com/hexjelly) (platforms: Windows, AMD)\n\n## Notes\n#### Code quality review\nIf you'd like to evaluate the quality of this library, you may find audit comments helpful.\n\nSimply search for \"AUDIT\" to see the dev notes that are aimed at facilitating code reviews.\n\n#### Implementation notes\nThe `1.X.X` implementation mostly copies [BackBlaze's Java implementation](https://github.com/Backblaze/JavaReedSolomon).\n\n`2.0.0` onward mostly copies [Klaus Post's Go implementation](https://github.com/klauspost/reedsolomon), and copies C files from [Nicolas Trangez's Haskell implementation](https://github.com/NicolasT/reedsolomon).\n\nThe test suite for all versions copies [Klaus Post's Go implementation](https://github.com/klauspost/reedsolomon) as basis.\n\n## License\n#### Nicolas Trangez's Haskell Reed-Solomon implementation\nThe C files for SIMD operations are copied (with no/minor modifications) from [Nicolas Trangez's Haskell implementation](https://github.com/NicolasT/reedsolomon), and are under the same MIT License as used by NicolasT's project\n\n#### TL;DR\nAll files are released under the MIT License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frust-rse%2Freed-solomon-erasure","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frust-rse%2Freed-solomon-erasure","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frust-rse%2Freed-solomon-erasure/lists"}